{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "J0Qjg6vuaHNt"
   },
   "source": [
    "# TensorFlow2.0教程-注意力神经机器翻译"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CiwtNgENbx2g"
   },
   "source": [
    "训练序列到序列（seq2seq）模型西班牙语到英语翻译。这是一个高级示例，假定序列模型的序列知识。\n",
    "\n",
    "在本教程中能够输入西班牙语句子，例如“¿todavia estan en casa？” ，并返回英文翻译：“你还在家吗？”\n",
    "\n",
    "对于玩具示例而言，翻译质量是合理的，但生成的关注图可能更有趣。这表明输入句子的哪些部分在翻译时具有模型的注意力：\n",
    "\n",
    "<img src=\"https://tensorflow.org/images/spanish-english.png\" alt=\"spanish-english attention plot\">\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 415
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 10605,
     "status": "ok",
     "timestamp": 1564762374047,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "tnxXKDjq3jEL",
    "outputId": "d4c18730-6b30-40b0-bd7d-1cbc184f52a3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: tensorflow-gpu==2.0.0-beta1 in /usr/local/lib/python3.6/dist-packages (2.0.0b1)\n",
      "Requirement already satisfied: tf-estimator-nightly<1.14.0.dev2019060502,>=1.14.0.dev2019060501 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (1.14.0.dev2019060501)\n",
      "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (1.11.2)\n",
      "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (1.15.0)\n",
      "Requirement already satisfied: tb-nightly<1.14.0a20190604,>=1.14.0a20190603 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (1.14.0a20190603)\n",
      "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (1.1.0)\n",
      "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (0.33.4)\n",
      "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (0.7.1)\n",
      "Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (1.16.4)\n",
      "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (3.7.1)\n",
      "Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (1.0.8)\n",
      "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (0.8.0)\n",
      "Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (0.2.2)\n",
      "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (1.1.0)\n",
      "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (0.1.7)\n",
      "Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-beta1) (1.12.0)\n",
      "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow-gpu==2.0.0-beta1) (0.15.5)\n",
      "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow-gpu==2.0.0-beta1) (41.0.1)\n",
      "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow-gpu==2.0.0-beta1) (3.1.1)\n",
      "Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.6->tensorflow-gpu==2.0.0-beta1) (2.8.0)\n",
      "2.0.0-beta1\n"
     ]
    }
   ],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "\n",
    "!pip install tensorflow-gpu==2.0.0-beta1\n",
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "import unicodedata\n",
    "import re\n",
    "import numpy as np\n",
    "import os\n",
    "import io\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wfodePkj3jEa"
   },
   "source": [
    "## 下载并准备数据集\n",
    "我们将使用http://www.manythings.org/anki/提供的语言数据集。此数据集包含以下格式的语言翻译对：\n",
    "\n",
    "May I borrow this book? ¿Puedo tomar prestado este libro?\n",
    "\n",
    "有多种语言可供选择，但我们将使用英语 - 西班牙语数据集。为方便起见，我们在Google Cloud上托管了此数据集的副本，但您也可以下载自己的副本。下载数据集后，以下是我们准备数据的步骤：\n",
    "\n",
    "- 为每个句子添加开始和结束标记。\n",
    "- 删除特殊字符来清除句子。\n",
    "- 创建一个单词索引和反向单词索引（从单词→id和id→单词映射的字典）。\n",
    "- 将每个句子填充到最大长度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kRVATYOgJs1b"
   },
   "outputs": [],
   "source": [
    "# Download the file\n",
    "path_to_zip = tf.keras.utils.get_file(\n",
    "    'spa-eng.zip', origin='http://storage.googleapis.com/download.tensorflow.org/data/spa-eng.zip',\n",
    "    extract=True)\n",
    "\n",
    "path_to_file = os.path.dirname(path_to_zip)+\"/spa-eng/spa.txt\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rd0jw-eC3jEh"
   },
   "outputs": [],
   "source": [
    "# Converts the unicode file to ascii\n",
    "def unicode_to_ascii(s):\n",
    "    return ''.join(c for c in unicodedata.normalize('NFD', s)\n",
    "        if unicodedata.category(c) != 'Mn')\n",
    "\n",
    "\n",
    "def preprocess_sentence(w):\n",
    "    w = unicode_to_ascii(w.lower().strip())\n",
    "\n",
    "    # creating a space between a word and the punctuation following it\n",
    "    # eg: \"he is a boy.\" => \"he is a boy .\"\n",
    "    # Reference:- https://stackoverflow.com/questions/3645931/python-padding-punctuation-with-white-spaces-keeping-punctuation\n",
    "    w = re.sub(r\"([?.!,¿])\", r\" \\1 \", w)\n",
    "    w = re.sub(r'[\" \"]+', \" \", w)\n",
    "\n",
    "    # replacing everything with space except (a-z, A-Z, \".\", \"?\", \"!\", \",\")\n",
    "    w = re.sub(r\"[^a-zA-Z?.!,¿]+\", \" \", w)\n",
    "\n",
    "    w = w.rstrip().strip()\n",
    "\n",
    "    # adding a start and an end token to the sentence\n",
    "    # so that the model know when to start and stop predicting.\n",
    "    w = '<start> ' + w + ' <end>'\n",
    "    return w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 10574,
     "status": "ok",
     "timestamp": 1564762374049,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "opI2GzOt479E",
    "outputId": "9597fe66-d08c-42a2-a9cb-2d21d72cc240"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<start> may i borrow this book ? <end>\n",
      "b'<start> \\xc2\\xbf puedo tomar prestado este libro ? <end>'\n"
     ]
    }
   ],
   "source": [
    "en_sentence = u\"May I borrow this book?\"\n",
    "sp_sentence = u\"¿Puedo tomar prestado este libro?\"\n",
    "print(preprocess_sentence(en_sentence))\n",
    "print(preprocess_sentence(sp_sentence).encode('utf-8'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OHn4Dct23jEm"
   },
   "outputs": [],
   "source": [
    "# 1. Remove the accents\n",
    "# 2. Clean the sentences\n",
    "# 3. Return word pairs in the format: [ENGLISH, SPANISH]\n",
    "def create_dataset(path, num_examples):\n",
    "    lines = io.open(path, encoding='UTF-8').read().strip().split('\\n')\n",
    "\n",
    "    word_pairs = [[preprocess_sentence(w) for w in l.split('\\t')]  for l in lines[:num_examples]]\n",
    "\n",
    "    return zip(*word_pairs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 73
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 15504,
     "status": "ok",
     "timestamp": 1564762379003,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "cTbSbBz55QtF",
    "outputId": "b12a315a-a645-4cae-a1f5-1cd7715719e3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<start> if you want to sound like a native speaker , you must be willing to practice saying the same sentence over and over in the same way that banjo players practice the same phrase over and over until they can play it correctly and at the desired tempo . <end>\n",
      "<start> si quieres sonar como un hablante nativo , debes estar dispuesto a practicar diciendo la misma frase una y otra vez de la misma manera en que un musico de banjo practica el mismo fraseo una y otra vez hasta que lo puedan tocar correctamente y en el tiempo esperado . <end>\n"
     ]
    }
   ],
   "source": [
    "en, sp = create_dataset(path_to_file, None)\n",
    "print(en[-1])\n",
    "print(sp[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OmMZQpdO60dt"
   },
   "outputs": [],
   "source": [
    "def max_length(tensor):\n",
    "    return max(len(t) for t in tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bIOn8RCNDJXG"
   },
   "outputs": [],
   "source": [
    "def tokenize(lang):\n",
    "  lang_tokenizer = tf.keras.preprocessing.text.Tokenizer(\n",
    "      filters='')\n",
    "  lang_tokenizer.fit_on_texts(lang)\n",
    "\n",
    "  tensor = lang_tokenizer.texts_to_sequences(lang)\n",
    "\n",
    "  tensor = tf.keras.preprocessing.sequence.pad_sequences(tensor,\n",
    "                                                         padding='post')\n",
    "\n",
    "  return tensor, lang_tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "eAY9k49G3jE_"
   },
   "outputs": [],
   "source": [
    "def load_dataset(path, num_examples=None):\n",
    "    # creating cleaned input, output pairs\n",
    "    targ_lang, inp_lang = create_dataset(path, num_examples)\n",
    "\n",
    "    input_tensor, inp_lang_tokenizer = tokenize(inp_lang)\n",
    "    target_tensor, targ_lang_tokenizer = tokenize(targ_lang)\n",
    "\n",
    "    return input_tensor, target_tensor, inp_lang_tokenizer, targ_lang_tokenizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GOi42V79Ydlr"
   },
   "source": [
    "### 限制数据集的大小以更快地进行实验（可选）\n",
    "对> 100,000个句子的完整数据集进行培训需要很长时间。为了更快地训练，我们可以将数据集的大小限制为30,000个句子（当然，翻译质量会随着数据的减少而降低）：\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cnxC7q-j3jFD"
   },
   "outputs": [],
   "source": [
    "# Try experimenting with the size of that dataset\n",
    "num_examples = 30000\n",
    "input_tensor, target_tensor, inp_lang, targ_lang = load_dataset(path_to_file, num_examples)\n",
    "\n",
    "# Calculate max_length of the target tensors\n",
    "max_length_targ, max_length_inp = max_length(target_tensor), max_length(input_tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 18029,
     "status": "ok",
     "timestamp": 1564762381576,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "4QILQkOs3jFG",
    "outputId": "a63f393f-6e7a-4650-8476-159d80fecd7b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24000 24000 6000 6000\n"
     ]
    }
   ],
   "source": [
    "# Creating training and validation sets using an 80-20 split\n",
    "input_tensor_train, input_tensor_val, target_tensor_train, target_tensor_val = train_test_split(input_tensor, target_tensor, test_size=0.2)\n",
    "\n",
    "# Show length\n",
    "print(len(input_tensor_train), len(target_tensor_train), len(input_tensor_val), len(target_tensor_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "lJPmLZGMeD5q"
   },
   "outputs": [],
   "source": [
    "def convert(lang, tensor):\n",
    "  for t in tensor:\n",
    "    if t!=0:\n",
    "      print (\"%d ----> %s\" % (t, lang.index_word[t]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 18013,
     "status": "ok",
     "timestamp": 1564762381579,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "VXukARTDd7MT",
    "outputId": "e8406fbb-6ba6-4d75-cd5e-ca1d5948d0ff"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input Language; index to word mapping\n",
      "1 ----> <start>\n",
      "6 ----> ¿\n",
      "52 ----> eres\n",
      "3361 ----> aficionado\n",
      "70 ----> al\n",
      "911 ----> golf\n",
      "5 ----> ?\n",
      "2 ----> <end>\n",
      "\n",
      "Target Language; index to word mapping\n",
      "1 ----> <start>\n",
      "24 ----> are\n",
      "6 ----> you\n",
      "1089 ----> fond\n",
      "59 ----> of\n",
      "809 ----> golf\n",
      "7 ----> ?\n",
      "2 ----> <end>\n"
     ]
    }
   ],
   "source": [
    "print (\"Input Language; index to word mapping\")\n",
    "convert(inp_lang, input_tensor_train[0])\n",
    "print ()\n",
    "print (\"Target Language; index to word mapping\")\n",
    "convert(targ_lang, target_tensor_train[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rgCLkfv5uO3d"
   },
   "source": [
    "### 创建tf.data数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "TqHsArVZ3jFS"
   },
   "outputs": [],
   "source": [
    "BUFFER_SIZE = len(input_tensor_train)\n",
    "BATCH_SIZE = 64\n",
    "steps_per_epoch = len(input_tensor_train)//BATCH_SIZE\n",
    "embedding_dim = 256\n",
    "units = 1024\n",
    "vocab_inp_size = len(inp_lang.word_index)+1\n",
    "vocab_tar_size = len(targ_lang.word_index)+1\n",
    "\n",
    "dataset = tf.data.Dataset.from_tensor_slices((input_tensor_train, target_tensor_train)).shuffle(BUFFER_SIZE)\n",
    "dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 17993,
     "status": "ok",
     "timestamp": 1564762381580,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "qc6-NK1GtWQt",
    "outputId": "49776563-8a3c-4d28-a23f-2aaaa1bb59a0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(TensorShape([64, 16]), TensorShape([64, 11]))"
      ]
     },
     "execution_count": 52,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_input_batch, example_target_batch = next(iter(dataset))\n",
    "example_input_batch.shape, example_target_batch.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TNfHIF71ulLu"
   },
   "source": [
    "编写编码器和解码器模型\n",
    "实现编码器 - 解码器模型，您可以在TensorFlow 神经机器翻译（seq2seq）教程中阅读。此示例使用更新的API集。这个笔记本实现了seq2seq教程中的注意方程式。下图显示了每个输入单词由注意机制分配权重，然后解码器使用该权重来预测句子中的下一个单词。下面的图片和公式是Luong的论文中关注机制的一个例子。\n",
    "\n",
    "<img src=\"https://www.tensorflow.org/images/seq2seq/attention_mechanism.jpg\" width=\"500\" alt=\"attention mechanism\">\n",
    "\n",
    "The input is put through an encoder model which gives us the encoder output of shape *(batch_size, max_length, hidden_size)* and the encoder hidden state of shape *(batch_size, hidden_size)*.\n",
    "\n",
    "Here are the equations that are implemented:\n",
    "\n",
    "<img src=\"https://www.tensorflow.org/images/seq2seq/attention_equation_0.jpg\" alt=\"attention equation 0\" width=\"800\">\n",
    "<img src=\"https://www.tensorflow.org/images/seq2seq/attention_equation_1.jpg\" alt=\"attention equation 1\" width=\"800\">\n",
    "\n",
    "本教程使用Bahdanau注意编码器。在编写简化表单之前，让我们决定表示法：\n",
    "\n",
    "- FC =完全连接（密集）层\n",
    "- EO =编码器输出\n",
    "- H =隐藏状态\n",
    "- X =解码器的输入\n",
    "和伪代码：\n",
    "\n",
    "- score = FC(tanh(FC(EO) + FC(H)))\n",
    "- attention weights = softmax(score, axis = 1)。默认情况下，Softmax应用于最后一个轴，但是我们要在第一个轴上应用它，因为得分的形状是（batch_size，max_length，hidden_​​size）。Max_length是我们输入的长度。由于我们尝试为每个输入分配权重，因此应在该轴上应用softmax。\n",
    "- context vector = sum(attention weights * EO, axis = 1)。选择轴为1的原因与上述相同。\n",
    "- embedding output =解码器X的输入通过嵌入层。\n",
    "- merged vector = concat(embedding output, context vector)\n",
    "- 然后将该合并的矢量提供给GRU\n",
    "每个步骤中所有向量的形状都已在代码中的注释中指定："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nZ2rI24i3jFg"
   },
   "outputs": [],
   "source": [
    "class Encoder(tf.keras.Model):\n",
    "  def __init__(self, vocab_size, embedding_dim, enc_units, batch_sz):\n",
    "    super(Encoder, self).__init__()\n",
    "    self.batch_sz = batch_sz\n",
    "    self.enc_units = enc_units\n",
    "    self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)\n",
    "    self.gru = tf.keras.layers.GRU(self.enc_units,\n",
    "                                   return_sequences=True,\n",
    "                                   return_state=True,\n",
    "                                   recurrent_initializer='glorot_uniform')\n",
    "\n",
    "  def call(self, x, hidden):\n",
    "    x = self.embedding(x)\n",
    "    output, state = self.gru(x, initial_state = hidden)\n",
    "    return output, state\n",
    "\n",
    "  def initialize_hidden_state(self):\n",
    "    return tf.zeros((self.batch_sz, self.enc_units))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 17975,
     "status": "ok",
     "timestamp": 1564762381581,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "60gSVh05Jl6l",
    "outputId": "4718a2ce-5217-4dfd-ea27-1e56fcbfb975"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Encoder output shape: (batch size, sequence length, units) (64, 16, 1024)\n",
      "Encoder Hidden state shape: (batch size, units) (64, 1024)\n"
     ]
    }
   ],
   "source": [
    "encoder = Encoder(vocab_inp_size, embedding_dim, units, BATCH_SIZE)\n",
    "\n",
    "# sample input\n",
    "sample_hidden = encoder.initialize_hidden_state()\n",
    "sample_output, sample_hidden = encoder(example_input_batch, sample_hidden)\n",
    "print ('Encoder output shape: (batch size, sequence length, units) {}'.format(sample_output.shape))\n",
    "print ('Encoder Hidden state shape: (batch size, units) {}'.format(sample_hidden.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "umohpBN2OM94"
   },
   "outputs": [],
   "source": [
    "class BahdanauAttention(tf.keras.Model):\n",
    "  def __init__(self, units):\n",
    "    super(BahdanauAttention, self).__init__()\n",
    "    self.W1 = tf.keras.layers.Dense(units)\n",
    "    self.W2 = tf.keras.layers.Dense(units)\n",
    "    self.V = tf.keras.layers.Dense(1)\n",
    "\n",
    "  def call(self, query, values):\n",
    "    # hidden shape == (batch_size, hidden size)\n",
    "    # hidden_with_time_axis shape == (batch_size, 1, hidden size)\n",
    "    # we are doing this to perform addition to calculate the score\n",
    "    hidden_with_time_axis = tf.expand_dims(query, 1)\n",
    "\n",
    "    # score shape == (batch_size, max_length, 1)\n",
    "    # we get 1 at the last axis because we are applying score to self.V\n",
    "    # the shape of the tensor before applying self.V is (batch_size, max_length, units)\n",
    "    score = self.V(tf.nn.tanh(\n",
    "        self.W1(values) + self.W2(hidden_with_time_axis)))\n",
    "\n",
    "    # attention_weights shape == (batch_size, max_length, 1)\n",
    "    attention_weights = tf.nn.softmax(score, axis=1)\n",
    "\n",
    "    # context_vector shape after sum == (batch_size, hidden_size)\n",
    "    context_vector = attention_weights * values\n",
    "    context_vector = tf.reduce_sum(context_vector, axis=1)\n",
    "\n",
    "    return context_vector, attention_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 17957,
     "status": "ok",
     "timestamp": 1564762381582,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "k534zTHiDjQU",
    "outputId": "164cb62a-4f36-4d80-e6f5-f1c0f67f3b04"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attention result shape: (batch size, units) (64, 1024)\n",
      "Attention weights shape: (batch_size, sequence_length, 1) (64, 16, 1)\n"
     ]
    }
   ],
   "source": [
    "attention_layer = BahdanauAttention(10)\n",
    "attention_result, attention_weights = attention_layer(sample_hidden, sample_output)\n",
    "\n",
    "print(\"Attention result shape: (batch size, units) {}\".format(attention_result.shape))\n",
    "print(\"Attention weights shape: (batch_size, sequence_length, 1) {}\".format(attention_weights.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yJ_B3mhW3jFk"
   },
   "outputs": [],
   "source": [
    "class Decoder(tf.keras.Model):\n",
    "  def __init__(self, vocab_size, embedding_dim, dec_units, batch_sz):\n",
    "    super(Decoder, self).__init__()\n",
    "    self.batch_sz = batch_sz\n",
    "    self.dec_units = dec_units\n",
    "    self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)\n",
    "    self.gru = tf.keras.layers.GRU(self.dec_units,\n",
    "                                   return_sequences=True,\n",
    "                                   return_state=True,\n",
    "                                   recurrent_initializer='glorot_uniform')\n",
    "    self.fc = tf.keras.layers.Dense(vocab_size)\n",
    "\n",
    "    # used for attention\n",
    "    self.attention = BahdanauAttention(self.dec_units)\n",
    "\n",
    "  def call(self, x, hidden, enc_output):\n",
    "    # enc_output shape == (batch_size, max_length, hidden_size)\n",
    "    context_vector, attention_weights = self.attention(hidden, enc_output)\n",
    "\n",
    "    # x shape after passing through embedding == (batch_size, 1, embedding_dim)\n",
    "    x = self.embedding(x)\n",
    "\n",
    "    # x shape after concatenation == (batch_size, 1, embedding_dim + hidden_size)\n",
    "    x = tf.concat([tf.expand_dims(context_vector, 1), x], axis=-1)\n",
    "\n",
    "    # passing the concatenated vector to the GRU\n",
    "    output, state = self.gru(x)\n",
    "\n",
    "    # output shape == (batch_size * 1, hidden_size)\n",
    "    output = tf.reshape(output, (-1, output.shape[2]))\n",
    "\n",
    "    # output shape == (batch_size, vocab)\n",
    "    x = self.fc(output)\n",
    "\n",
    "    return x, state, attention_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 17938,
     "status": "ok",
     "timestamp": 1564762381582,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "P5UY8wko3jFp",
    "outputId": "9da1a49a-181d-48c9-f87a-b5f3719f6a7d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Decoder output shape: (batch_size, vocab size) (64, 4935)\n"
     ]
    }
   ],
   "source": [
    "decoder = Decoder(vocab_tar_size, embedding_dim, units, BATCH_SIZE)\n",
    "\n",
    "sample_decoder_output, _, _ = decoder(tf.random.uniform((64, 1)),\n",
    "                                      sample_hidden, sample_output)\n",
    "\n",
    "print ('Decoder output shape: (batch_size, vocab size) {}'.format(sample_decoder_output.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_ch_71VbIRfK"
   },
   "source": [
    "## 定义优化器和损失函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WmTHr5iV3jFr"
   },
   "outputs": [],
   "source": [
    "optimizer = tf.keras.optimizers.Adam()\n",
    "loss_object = tf.keras.losses.SparseCategoricalCrossentropy(\n",
    "    from_logits=True, reduction='none')\n",
    "\n",
    "def loss_function(real, pred):\n",
    "  mask = tf.math.logical_not(tf.math.equal(real, 0))\n",
    "  loss_ = loss_object(real, pred)\n",
    "\n",
    "  mask = tf.cast(mask, dtype=loss_.dtype)\n",
    "  loss_ *= mask\n",
    "\n",
    "  return tf.reduce_mean(loss_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DMVWzzsfNl4e"
   },
   "source": [
    "## 检查点（基于对象的保存）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Zj8bXQTgNwrF"
   },
   "outputs": [],
   "source": [
    "checkpoint_dir = './training_checkpoints'\n",
    "checkpoint_prefix = os.path.join(checkpoint_dir, \"ckpt\")\n",
    "checkpoint = tf.train.Checkpoint(optimizer=optimizer,\n",
    "                                 encoder=encoder,\n",
    "                                 decoder=decoder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hpObfY22IddU"
   },
   "source": [
    "## 训练\n",
    "- 通过编码器传递输入，编码器返回编码器输出和编码器隐藏状态。\n",
    "- 编码器输出，编码器隐藏状态和解码器输入（它是开始标记）被传递给解码器。\n",
    "- 解码器返回预测和解码器隐藏状态。\n",
    "- 然后将解码器隐藏状态传递回模型，并使用预测来计算损失。\n",
    "- 使用教师强制决定解码器的下一个输入。\n",
    "- 教师强制是将目标字作为下一个输入传递给解码器的技术。\n",
    "- 最后一步是计算渐变并将其应用于优化器并反向传播。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sC9ArXSsVfqn"
   },
   "outputs": [],
   "source": [
    "@tf.function\n",
    "def train_step(inp, targ, enc_hidden):\n",
    "  loss = 0\n",
    "\n",
    "  with tf.GradientTape() as tape:\n",
    "    enc_output, enc_hidden = encoder(inp, enc_hidden)\n",
    "\n",
    "    dec_hidden = enc_hidden\n",
    "\n",
    "    dec_input = tf.expand_dims([targ_lang.word_index['<start>']] * BATCH_SIZE, 1)\n",
    "\n",
    "    # Teacher forcing - feeding the target as the next input\n",
    "    for t in range(1, targ.shape[1]):\n",
    "      # passing enc_output to the decoder\n",
    "      predictions, dec_hidden, _ = decoder(dec_input, dec_hidden, enc_output)\n",
    "\n",
    "      loss += loss_function(targ[:, t], predictions)\n",
    "\n",
    "      # using teacher forcing\n",
    "      dec_input = tf.expand_dims(targ[:, t], 1)\n",
    "\n",
    "  batch_loss = (loss / int(targ.shape[1]))\n",
    "\n",
    "  variables = encoder.trainable_variables + decoder.trainable_variables\n",
    "\n",
    "  gradients = tape.gradient(loss, variables)\n",
    "\n",
    "  optimizer.apply_gradients(zip(gradients, variables))\n",
    "\n",
    "  return batch_loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 457194,
     "status": "ok",
     "timestamp": 1564762820867,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "ddefjBMa3jF0",
    "outputId": "f1966e2e-dd6e-4a83-b1ca-ae5d4b26d905"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 Batch 0 Loss 4.6147\n",
      "Epoch 1 Batch 100 Loss 2.1792\n",
      "Epoch 1 Batch 200 Loss 1.8256\n",
      "Epoch 1 Batch 300 Loss 1.7484\n",
      "Epoch 1 Loss 2.0445\n",
      "Time taken for 1 epoch 65.14264225959778 sec\n",
      "\n",
      "Epoch 2 Batch 0 Loss 1.7078\n",
      "Epoch 2 Batch 100 Loss 1.5201\n",
      "Epoch 2 Batch 200 Loss 1.3324\n",
      "Epoch 2 Batch 300 Loss 1.2268\n",
      "Epoch 2 Loss 1.4118\n",
      "Time taken for 1 epoch 41.88753700256348 sec\n",
      "\n",
      "Epoch 3 Batch 0 Loss 1.3513\n",
      "Epoch 3 Batch 100 Loss 1.0342\n",
      "Epoch 3 Batch 200 Loss 0.9434\n",
      "Epoch 3 Batch 300 Loss 0.8773\n",
      "Epoch 3 Loss 1.0038\n",
      "Time taken for 1 epoch 41.1470890045166 sec\n",
      "\n",
      "Epoch 4 Batch 0 Loss 0.9539\n",
      "Epoch 4 Batch 100 Loss 0.6765\n",
      "Epoch 4 Batch 200 Loss 0.6487\n",
      "Epoch 4 Batch 300 Loss 0.5726\n",
      "Epoch 4 Loss 0.6843\n",
      "Time taken for 1 epoch 41.780235052108765 sec\n",
      "\n",
      "Epoch 5 Batch 0 Loss 0.6601\n",
      "Epoch 5 Batch 100 Loss 0.4600\n",
      "Epoch 5 Batch 200 Loss 0.4444\n",
      "Epoch 5 Batch 300 Loss 0.3771\n",
      "Epoch 5 Loss 0.4654\n",
      "Time taken for 1 epoch 41.11035752296448 sec\n",
      "\n",
      "Epoch 6 Batch 0 Loss 0.4556\n",
      "Epoch 6 Batch 100 Loss 0.3045\n",
      "Epoch 6 Batch 200 Loss 0.3213\n",
      "Epoch 6 Batch 300 Loss 0.2689\n",
      "Epoch 6 Loss 0.3225\n",
      "Time taken for 1 epoch 41.67611289024353 sec\n",
      "\n",
      "Epoch 7 Batch 0 Loss 0.3330\n",
      "Epoch 7 Batch 100 Loss 0.1870\n",
      "Epoch 7 Batch 200 Loss 0.1904\n",
      "Epoch 7 Batch 300 Loss 0.1787\n",
      "Epoch 7 Loss 0.2293\n",
      "Time taken for 1 epoch 41.24461650848389 sec\n",
      "\n",
      "Epoch 8 Batch 0 Loss 0.2096\n",
      "Epoch 8 Batch 100 Loss 0.1414\n",
      "Epoch 8 Batch 200 Loss 0.1424\n",
      "Epoch 8 Batch 300 Loss 0.1320\n",
      "Epoch 8 Loss 0.1658\n",
      "Time taken for 1 epoch 41.80285120010376 sec\n",
      "\n",
      "Epoch 9 Batch 0 Loss 0.1480\n",
      "Epoch 9 Batch 100 Loss 0.0893\n",
      "Epoch 9 Batch 200 Loss 0.1017\n",
      "Epoch 9 Batch 300 Loss 0.0940\n",
      "Epoch 9 Loss 0.1245\n",
      "Time taken for 1 epoch 41.111262798309326 sec\n",
      "\n",
      "Epoch 10 Batch 0 Loss 0.1155\n",
      "Epoch 10 Batch 100 Loss 0.0605\n",
      "Epoch 10 Batch 200 Loss 0.1058\n",
      "Epoch 10 Batch 300 Loss 0.0665\n",
      "Epoch 10 Loss 0.0972\n",
      "Time taken for 1 epoch 41.78199052810669 sec\n",
      "\n"
     ]
    }
   ],
   "source": [
    "EPOCHS = 10\n",
    "\n",
    "for epoch in range(EPOCHS):\n",
    "  start = time.time()\n",
    "\n",
    "  enc_hidden = encoder.initialize_hidden_state()\n",
    "  total_loss = 0\n",
    "\n",
    "  for (batch, (inp, targ)) in enumerate(dataset.take(steps_per_epoch)):\n",
    "    batch_loss = train_step(inp, targ, enc_hidden)\n",
    "    total_loss += batch_loss\n",
    "\n",
    "    if batch % 100 == 0:\n",
    "        print('Epoch {} Batch {} Loss {:.4f}'.format(epoch + 1,\n",
    "                                                     batch,\n",
    "                                                     batch_loss.numpy()))\n",
    "  # saving (checkpoint) the model every 2 epochs\n",
    "  if (epoch + 1) % 2 == 0:\n",
    "    checkpoint.save(file_prefix = checkpoint_prefix)\n",
    "\n",
    "  print('Epoch {} Loss {:.4f}'.format(epoch + 1,\n",
    "                                      total_loss / steps_per_epoch))\n",
    "  print('Time taken for 1 epoch {} sec\\n'.format(time.time() - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mU3Ce8M6I3rz"
   },
   "source": [
    "## 翻译\n",
    "- evaluate函数类似于训练循环，除了我们在这里不使用教师强制。在每个时间步骤对解码器的输入是其先前的预测以及隐藏状态和编码器输出。\n",
    "- 停止预测模型何时预测结束标记。\n",
    "- 并存储每个时间步的注意力。\n",
    "注意：编码器输出仅针对一个输入计算一次。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "EbQpyYs13jF_"
   },
   "outputs": [],
   "source": [
    "def evaluate(sentence):\n",
    "    attention_plot = np.zeros((max_length_targ, max_length_inp))\n",
    "\n",
    "    sentence = preprocess_sentence(sentence)\n",
    "\n",
    "    inputs = [inp_lang.word_index[i] for i in sentence.split(' ')]\n",
    "    inputs = tf.keras.preprocessing.sequence.pad_sequences([inputs],\n",
    "                                                           maxlen=max_length_inp,\n",
    "                                                           padding='post')\n",
    "    inputs = tf.convert_to_tensor(inputs)\n",
    "\n",
    "    result = ''\n",
    "\n",
    "    hidden = [tf.zeros((1, units))]\n",
    "    enc_out, enc_hidden = encoder(inputs, hidden)\n",
    "\n",
    "    dec_hidden = enc_hidden\n",
    "    dec_input = tf.expand_dims([targ_lang.word_index['<start>']], 0)\n",
    "\n",
    "    for t in range(max_length_targ):\n",
    "        predictions, dec_hidden, attention_weights = decoder(dec_input,\n",
    "                                                             dec_hidden,\n",
    "                                                             enc_out)\n",
    "\n",
    "        # storing the attention weights to plot later on\n",
    "        attention_weights = tf.reshape(attention_weights, (-1, ))\n",
    "        attention_plot[t] = attention_weights.numpy()\n",
    "\n",
    "        predicted_id = tf.argmax(predictions[0]).numpy()\n",
    "\n",
    "        result += targ_lang.index_word[predicted_id] + ' '\n",
    "\n",
    "        if targ_lang.index_word[predicted_id] == '<end>':\n",
    "            return result, sentence, attention_plot\n",
    "\n",
    "        # the predicted ID is fed back into the model\n",
    "        dec_input = tf.expand_dims([predicted_id], 0)\n",
    "\n",
    "    return result, sentence, attention_plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "s5hQWlbN3jGF"
   },
   "outputs": [],
   "source": [
    "# function for plotting the attention weights\n",
    "def plot_attention(attention, sentence, predicted_sentence):\n",
    "    fig = plt.figure(figsize=(10,10))\n",
    "    ax = fig.add_subplot(1, 1, 1)\n",
    "    ax.matshow(attention, cmap='viridis')\n",
    "\n",
    "    fontdict = {'fontsize': 14}\n",
    "\n",
    "    ax.set_xticklabels([''] + sentence, fontdict=fontdict, rotation=90)\n",
    "    ax.set_yticklabels([''] + predicted_sentence, fontdict=fontdict)\n",
    "\n",
    "    ax.xaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "    ax.yaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sl9zUHzg3jGI"
   },
   "outputs": [],
   "source": [
    "def translate(sentence):\n",
    "    result, sentence, attention_plot = evaluate(sentence)\n",
    "\n",
    "    print('Input: %s' % (sentence))\n",
    "    print('Predicted translation: {}'.format(result))\n",
    "\n",
    "    attention_plot = attention_plot[:len(result.split(' ')), :len(sentence.split(' '))]\n",
    "    plot_attention(attention_plot, sentence.split(' '), result.split(' '))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n250XbnjOaqP"
   },
   "source": [
    "## 恢复最新的检查点并进行测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 457912,
     "status": "ok",
     "timestamp": 1564762821673,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "UJpT9D5_OgP6",
    "outputId": "86ee1b5b-f73b-4a48-dcdb-8ed290147eb9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.training.tracking.util.CheckpointLoadStatus at 0x7f1e3e220e48>"
      ]
     },
     "execution_count": 66,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# restoring the latest checkpoint in checkpoint_dir\n",
    "checkpoint.restore(tf.train.latest_checkpoint(checkpoint_dir))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 677
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 458400,
     "status": "ok",
     "timestamp": 1564762822187,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "WrAM0FDomq3E",
    "outputId": "a5d67baf-a8b1-48b2-d6c9-01df7289fae1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> hace mucho frio aqui . <end>\n",
      "Predicted translation: it s very cold here . <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiYAAAJwCAYAAAC9GaqsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xm8rnVd7//3BzaDgGiKKXpCMQdw\nFnY4nsS0OGlZeZoUDbOfWOkvpzLNU5Lnp+ZUUTaIFf4QLNGjx+nkrDlnaKY4bxHUFJFCZR4/54/r\n3rLWcgMb2Htd37XW8/l47Mdjreu+170+62Kz7te+xuruAACMYJe5BwAA2EqYAADDECYAwDCECQAw\nDGECAAxDmAAAwxAmAMAwhAkAMAxhAgAMQ5gAAMMQJgOoqttX1bur6q5zzwIAcxImYzgqyeFJHjvz\nHAAwq3ITv3lVVSU5Pck7kvx0klt29+WzDgUAM7HFZH6HJ7lhkt9KclmSh8w6DQDMSJjM76gkr+3u\nC5L8w+JzANiQ7MqZUVXtneQbSR7a3e+vqnsk+XCS/bv72/NOBwCrzxaTef33JGd39/uTpLs/keSL\nSX551qkAWPOqau+q+pWqutHcs1wbwmRej05y4oplJyZ5zOqPAsA684tJjs/0XrNm2JUzk6r6oSRf\nTnJwd39xyfL/kuksnTt19xdmGo8BVNXdkvx2kjsl6SSfSfKi7j511sGANaGq3pPk5kku6O7Nc8+z\nvYQJDKiqHpbkdUnen+QDi8X3X/x5eHe/aa7ZgPFV1W2SfCHJYUk+kuSQ7v7MnDNtL2Eyo6o6IMlX\nexv/EarqgO7+ygxjMYCq+mSS13f3s1csf06Sn+nuu88zGbAWVNXvJzm8ux9UVa9L8sXu/t2559oe\njjGZ15eT3Gzlwqq66eIxNq47JHnlNpa/MskdV3kWYO35lVz5O+SkJEcuLug5PGEyr8p07MBK+yS5\naJVnYSxnJTl0G8sPTfLNVZ4FWEOq6r5J9k/y2sWiNyXZK8mDZxvqWtg09wAbUVX92eLDTvL8qrpg\nycO7Zton+IlVH4yRvDzJy6rqdkk+tFh2v0wHw75otqmAteCoJG/o7vOSpLsvqaqTM53x+Y45B9se\njjGZweJI6SR5QKYLql2y5OFLMp2V8+KlZ+uwsSw2uT45ydOS3HKx+OuZouTPtnVcEkBV7ZHkzCSP\n6O63Lll+/yRvS3LzrcEyKmEyk8Ubz8lJHtvd5849D+Oqqhsmib8nwDWpqv0y3XPtxO6+YsVjj0ry\nzu4+c5bhtpMwmUlV7ZrpOJK7r5VTuABgZ3OMyUy6+/KqOiPJ7nPPwniq6iZJnpvkQUl+MCsOVO/u\nfeeYC2BnEybz+p9J/qiqHtXdZ889DEP52yT3THJcpmNLbNoErlJVfTnb+Xuiu2+7k8e5XuzKmVFV\nfSrJgUl2S/K1JOcvfby77zbHXMyvqr6b5Me7+5/nngUYX1U9bcmn+yR5apKPZjrBIknuk+mMz5d0\n93NWebxrxRaTeb32mp/CBnVWkqGPnAfG0d0v2fpxVb0iyQu6+3lLn1NVz0xy51Ue7VqzxQQGVFW/\nlOnOoEeNfmofMJbFFtdDunvLiuW3S/Lx0Y9Rs8WEYVTVbyZ5QqbdW3fp7tOq6hlJTuvuk+edbudb\n7Npb+i+FA5OctThI+tKlz7WbD7ga5yc5PMmWFcsPT3LByiePRpjMqKp2T/KsJI9IckCmY02+p7t3\nnWOuOVTVk5M8PckLkvzRkof+PckTM13zZb2zaw/YEf4kyV9U1eZMdxZOkntnuiLsMXMNtb3syplR\nVb0gyS8leX6mv0j/I8ltkvxykt/v7pfNN93qqqrPJXlad7+lqs7NdH2X06rqzkne1903nXlEmFVV\nHZLkE919xeLjq9TdH1+lsRhUVf1ikiclOXix6LNJjl0LW5+FyYwWp3f9Rne/dfFmfI/u/lJV/UaS\nB3X3z8884qqpqguTHNTdZ6wIkztk+mW818wjrqqqekCSdPc/bWN5d/f7ZhmM2VTVFUlu0d1nLT7u\nTDcCXak30tZW1h+7cuZ18yRbr/p6XpIbLz5+a6ZdGhvJaUkOSXLGiuUPyZXraCP5kyTbOqVv30yb\nYrd152HWtwOTfGvJx3CNqurG+f4LNP7nTONsF2Eyr69kukHbVzIdpHREko9lOt/8whnnmsOLk7y0\nqvbK9K/A+1TVozMdd/LYWSebxx2T/Ns2lp+6eIwNprvP2NbHsFJV3TrJX2c62HXp1cUr05a2obeo\nCZN5vT7TJcc/kuTYJH9fVY9LcqtssFvbd/fxVbUpyfOS7JXklZmuePpb3f3qWYebx4VJ9k/y5RXL\nb5Xld6NmA3KMCdfg+Exb4H8ta/DK0Y4xGUhV3SvJ/ZJ8obvfPPc8c1ncHXOX7j5r7lnmUlUnZTpT\n62Hdfc5i2U2SvCHJ17r7EXPOx7yu4hiT7/0yd4zJxlZV5yW5d3efOvcs14UwmVFV/WiSD3X3ZSuW\nb0py3410gOPi7Jtdu/uTK5bfLcllG+0OzFW1f5L3ZbqB39Z1crdMV4R9QHd/fa7ZmN9iU/1Su2W6\nt9Kzkjyzu/9x9adiFItrIj2muz829yzXhTCZUVVdnmT/lVsGquqmSc7aSP/qqaoPJvmL7n7ViuW/\nnOSJ3X3/eSabz+J4myOT3GOx6F+TvKq7h79A0s5QVT+W5E6Ztgx8prvfM/NIw6mqn0jy7O6+39yz\nMJ/F/yvPSPKbK6/+uhYIkxktNsfevLu/tWL5HZKcMvplg3ekxSnC99zGJZR/ONMllG80z2TMrapu\nlel4rEMz7S9PpoPGT0nyc7YeXamqbp/p9Pq9556F+Sx+n+6R6SDXi5Ms2yo/+nuLg19nUFVvXHzY\nSU6sqouXPLxrkrsk+dCqDzavy5NsKz5+INu+VsO6VlUPv7rHu/t1qzXLAP4s09+P23X3l5Okqm6b\n5MTFYxvmej9bLY43WrYo08HSxyT5/KoPxGieOPcA14ctJjOoquMXHx6V6VLrS08NviTJ6Ule3t1n\nr/Jos6mqN2R68/mF7r58sWxTktck2a27f2rO+VbbYmvatnSysQ5uXNyQ7PCVZ5osLrf9ro24NW3J\nwa/LFif5apJf6u6PfP9Xwdpgi8kMuvtXk6SqTk/y4u4+f96JhvD0JB9IsqWqPrBYdv8k+yT50dmm\nmkl3L7sg0iLS7pnpNPJnzTLUvLb1L6iN/K+qB674/IpMF1/bsvJgejamqrp5kkcn+eFMtzg5u6ru\nl+TrW7c8jsoWkxlV1S5J0t1XLD6/RZKfynRg30bblbP1TJQnZvnBnn/pGIIrVdV9k/xVd9997llW\nS1W9PsnNkjyiu7+6WHZAkpOSfKu7r3a3F2w0VXVokndlug7SnTPd7uO0qjomyR26+5FzzndNhMmM\nquofk7y1u4+tqn2SfC7J3pm2Evxad58w64AMp6rulOSj3b3P3LOslqr6oSRvzHTs1dKDXz+V6Tov\nX5trtrksLjWwXTbSZQeYVNV7Mt389Nkr7j12nyT/0N0rTzcfil0589qcaRdGkjw8yXcz3QPjyCS/\nnWTDhUlV3TLThcWWXkZ5w/1y3caVPbce3Pi7mbYkbRjd/dXF+nhwkoMWiz/b3e+ccay5vTdX7sra\nenD4ys+3LtswxyPxPYdmuurrSt/IdI+2oQmTee2T5NuLj38iyeu7+9KqeneSv5hvrNW3CJJXZTqe\nZOsVLZduzttov1xPybbvHvuRbMB7B/W0afcdiz9Mu3xfnOS5ST68WHafJL+X6R87Dn7d2C7MdEbj\nSgdlukjj0ITJvL6S5H5V9aZMN/D7hcXymyTZaBfR+tNMZ+XcKcm/JPlvmcr+OUmeMuNcc1l599gr\nMh1PcdEcw6y2qnpqpuOLLlp8fJW6+49XaayR/M8kT+rupaF2WlWdleSF3X3PmeZiDG9I8uyq2vqe\n0lV1m0x3rf9fcw21vRxjMqOqenySlyY5L8kZSQ7p7iuq6reS/Gx3/9isA66iqvpmkod29ymL00M3\nd/cXquqhmY4ov/fMI666xVH198t0WfqVty3/y1mGWiVV9eVMfwf+Y/HxVenuvu1qzTWKqrow0++L\nz65YfqckH+vuG8wzGSOoqn2T/J9Mt7HYO8mZmf6h96EkPzn6maDCZGaLo6cPSPKO7j5vseyhSb7d\n3R+cdbhVtIiRu3X36YvTqB/V3R+oqgOTfLq795p3wtVVVY9K8jeZduWck+W7tbq7bznLYAyhqk5J\nsiXJr3b3hYtlN8h0V9nbdffmOedjDItL0x+S6R82H18rx2XZlTOTqrpRpjfi9ydZeaOlbyfZUDet\ny3RG0kGZLi73iSS/XlVfTfKEJP8+41xzeW6SFyZ5zka+LkVV7Zbp+ja/0t2uaHql30jy5iT/XlVb\nb/J410y7Qx8621TMbul7S3e/O8m7lzx2v0yXozhntgG3gy0mM6mqG2Y6QvqIpVtGquruST6a5FYb\n7MqvR2a6wusrFmdgvDXJfpnu83BUd58864CrrKrOSXJod5829yxzWxw3cf/u/sLcs4ykqvZO8sgk\nBy8WfTbTTR6H3kzPzrUe3luEyYyq6qQk53X345cse3GmC+A8bL7J5re4s+5BSb4y+v9EO0NVvTTJ\n57v7z+eeZW5V9aIk6e7fmXuWkSyuBnxYtn16/Ya71ABXWuvvLcJkRlV1RJK/T3KL7r5kcSXYryV5\n4ga7SVuSpKp+KcmDsu2DPYf/n2lHqqrdk/zvTPdO+lSSS5c+3t3PmWOuOVTVX2a6ts+XM+32XLZF\noLt/a4655lRVByV5U6aztyrTLpxNmf6eXDz63WPZudb6e4tjTOb1jkznm/9UktdlelPePdMvnA1l\n8a/iJyd5T6are270Yn58plOmz05yu6w4+DXTadTr1uLKph9aHF9zcJKtN/BbeQbORv178qeZIu0e\nmc64uEemu3P/VZL/MeNcjGFNv7fYYjKzqnpBkjt2989W1QlJzu3uJ8w912pbnC78hO5+7dyzjGBx\nXMXzu/tP5p5lDlV1eZL9u/usqjotyY9093/MPdcoquo/kjygu0+tqu8kOay7P19VD0jy5919t5lH\nZGZr+b3FFpP5nZDkY4ubkv1cprLdiHbJdDYOk10z3R9mozon026Ks5LcJit27ZHKlRdh/FaSWyX5\nfKbN9bebayiGsmbfW2wxGcDimgQXJtmvuw++puevR1X13CSXdvcxc88ygsWBat/dSMeSLFVVL0ty\nVKazCw7I9IZ7+baeu0EvsPa+JH/S3a+vqlcluWmS5yV5XKZTRW0xYc2+t9hiMoYTMu0zftbcg6ym\nqvqzJZ/ukuTIqvrxJJ/M9x/sudEOcNwryf+zOIhtI66PX8+0xej2Sf4404XDzp11orE8N9MVPZPp\nmJK3ZDo+6+wkvzjXUKOpqs8muX13b9T3ujX53rJR/2ON5sRMN1w6fu5BVtldV3y+dVfOQSuWb8TN\negfnyrsIb7j1sbhp31uS711/4SXdLUwWuvttSz4+LcnBVXWTJOe0zeBL/UWmrUkb1Zp8b7ErBwAY\nhgPKAIBhCBMAYBjCZBBVdfTcM4zE+ljO+ljO+ljO+ljO+lhura0PYTKONfUXZxVYH8tZH8tZH8tZ\nH8tZH8utqfUhTACAYWz4s3J2rz16z+9dDmA+l+bi7JY95h5jGNbHctbHctbHctbHctbHcqOsj3Nz\nztndfbNret6Gv47Jntk796o1c6VeAFiT3tmvPWN7nmdXDgAwDGECAAxDmAAAwxAmAMAwhAkAMAxh\nAgAMQ5gAAMMQJgDAMIQJADAMYQIADEOYAADDECYAwDCECQAwDGECAAxDmAAAwxAmAMAwhAkAMAxh\nAgAMQ5gAAMMQJgDAMIQJADAMYQIADEOYAADDECYAwDCECQAwDGECAAxDmAAAw1gXYVJVr6iqN889\nBwBw/Wyae4Ad5ElJKkmq6r1JTu3uJ846EQBwra2LMOnu78w9AwBw/a2LMKmqVyTZL8nZSR6Q5AFV\n9YTFwwd29+kzjQYAXAvrIkyWeFKSOyT5XJLfWyz71nzjAADXxroKk+7+TlVdkuSC7j7zqp5XVUcn\nOTpJ9sxeqzUeAHAN1sVZOddWdx/X3Zu7e/Nu2WPucQCAhQ0ZJgDAmNZjmFySZNe5hwAArr31GCan\nJzmsqm5TVftV1Xr8GQFgXVqPb9ovzrTV5DOZzsg5YN5xAIDttS7Oyunuxyz5+AtJ7jPfNADAdbUe\nt5gAAGuUMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBg\nGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBg\nGMIEABiGMAEAhrFp7gHmVnvsnl1vfdu5xxjGll+7+dwjDGW379TcIwzl1n/16blHGMoV550/9wgM\nrC+7bO4R1iRbTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjC\nBAAYhjABAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjC\nBAAYhjABAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhrHuwqSqfrSqPlJV\n51XVd6rqo1V1l7nnAgCu2aa5B9iRqmpTkjck+dskRybZLckhSS6fcy4AYPusqzBJsm+SGyd5U3d/\nabHscyufVFVHJzk6SfbctO/qTQcAXK11tSunu/8zySuSvK2q3lJVT62qA7bxvOO6e3N3b9591xus\n+pwAwLatqzBJku7+1ST3SvK+JA9L8vmqOmLeqQCA7bHuwiRJuvvfuvsF3X14kvcmOWreiQCA7bGu\nwqSqDqyqP6qq+1bVravqgUnuluQzc88GAFyz9Xbw6wVJ7pDkNUn2S/LNJCclecGcQwEA22ddhUl3\nfzPJw+eeAwC4btbVrhwAYG0TJgDAMIQJADAMYQIADEOYAADDECYAwDCECQAwDGECAAxDmAAAwxAm\nAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJADAMYQIADEOYAADDECYAwDCECQAwDGECAAxDmAAAwxAm\nAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMDbNPcDsLr88Oec7c08xjANft/fcIwzl9J/eZ+4RhvKdnzh4\n7hGGcqN/Om3uEcZyyaVzTzCUy7/97blHGEtv39NsMQEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACA\nYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACA\nYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACA\nYQgTAGAYaz5Mqmr3uWcAAHaMVQ2Tqjq6qr5ZVbuuWP6qqnrj4uOfrqqPVdVFVfXlqnru0vioqtOr\n6piq+ruq+naSk6rq3VX10hWvuW9VXVBVD1+VHw4AuN5We4vJa5LcKMmPb11QVfsk+ZkkJ1bVEUlO\nSvLSJHdO8tgkP5/keSte56lJPpdkc5LfS/LyJI+sqj2WPOcRSc5L8qad8pMAADvcqoZJd5+T5P8k\nOXLJ4p9NclmSNyZ5VpIXdffx3f2l7n5Pkt9N8utVVUu+5p+6+4XdvaW7v5jkdUmuSPJzS57z2CQn\ndPelK+dYbLk5papOueSKi3bozwgAXHdzHGNyYpKfraq9Fp8fmeR/dfdFSQ5N8qyqOm/rnySvSrJ3\nklsseY1Tlr5gd1+c5JWZYiRVdeckhyX5220N0N3Hdffm7t68+y577sAfDQC4PjbN8D3fkmkLyc9U\n1buSPDjJEYvHdknyh5l2+az0rSUfn7+Nx/8mySer6oBMgfLh7v7sDpsaANjpVj1MuvviqnpNpi0l\n+yU5M8l7Fw9/PMlB3b3lOrzup6vqn5M8LsmjMu0WAgDWkDm2mCTT7px3JTkwyd939xWL5c9J8uaq\nOiPJyZm2rNwlyWHd/fTteN2XJ/nrJJcmefUOnxoA2Knmuo7J+5P8e5I7ZYqUJEl3vy3JQ5M8MMlH\nF3+ekeQr2/m6r05ySZKTu/vcHTkwALDzzbLFpLs7yW2u4rG3J3n71XztNr9u4cZJbpCrOOgVABjb\nXLtydqiq2i3JTTNd7+Rfu/uDM48EAFwHa/6S9Av3S/KNJPfNdPArALAGrYstJt393iR1Tc8DAMa2\nXraYAADrgDABAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMA\nYBjCBAAYhjABAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhbJp7\ngLn1ZZfn8rP/Y+4xxmFdLHPbU/eae4ShHPDennuEoXzprIPnHmEoe2z55twjjOWcc+aeYE2yxQQA\nGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAYwgQA\nGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAYwgQA\nGIYwAQCGIUwAgGEIEwBgGMIEABjGmgyTqjqmqk69hue8tKreu0ojAQA7wJoMEwBgfRImAMAwZguT\nmjytqr5YVRdX1deq6vmLx+5aVe+sqgur6j+r6hVVdaOrea1dq+rFVXXO4s+fJtl11X4YAGCHmHOL\nyfOS/H6S5ye5c5JfSPLVqto7yduSnJfksCQ/l+S+Sf7ual7raUkel+TxSe6TKUqO3GmTAwA7xaY5\nvmlV7ZPkKUme3N1bg2NLkg9X1eOS7J3k0d197uL5Ryd5T1Xdrru3bOMln5zkhd198uL5T0pyxNV8\n/6OTHJ0ke2avHfRTAQDX11xbTO6UZI8k79rGYwcn+eTWKFn4UJIrFl+3zGIXz/5JPrx1WXdfkeSf\nr+qbd/dx3b25uzfvlj2u208AAOxwa+3g1557AABg55krTD6b5OIkD7qKx+5aVTdcsuy+mWb97Mon\nd/d3knwjyb23LquqynR8CgCwhsxyjEl3n1tVxyZ5flVdnOR9SW6a5NAk/3+SP0xyQlX9QZIfSPKy\nJK+7iuNLkuTYJM+sqi8k+VSS38y0e+cbO/cnAQB2pFnCZOGZSc7JdGbOf0nyzSQndPcFVXVEkj9N\n8tEkFyV5Q5InXc1rvSTJLZL8zeLzVyY5KdPxKgDAGjFbmCwOUP2jxZ+Vj30q297Ns/XxY5Ics+Tz\nyzKd5fOUHT0nALB61trBrwDAOiZMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAYwgQAGIYwAQCG\nIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAYwgQAGIYwAQCG\nIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhbJp7ABjZFRdcMPcIQzn93rvOPcJQdn/XmXOPMJRzXnbA\n3CMM5cbvuXTuEcaynf+72GICAAxDmAAAwxAmAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJADAMYQIA\nDEOYAADDECYAwDCECQAwDGECAAxDmAAAwxAmAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJADAMYQIA\nDEOYAADDECYAwDCECQAwDGECAAxDmAAAwxAmAMAwhAkAMAxhAgAMY9XCpKreW1UvXa3vBwCsPbaY\nAADDWNNhUlW7zT0DALDjrHaY7FJVz6uqs6vqrKp6cVXtkiRVtXtVvaCqvlZVF1TVv1TVEVu/sKoO\nr6quqodU1Uer6pIkRywe++mq+lhVXVRVX66q51bV7qv8swEA19OmVf5+RyY5Nsl9k9wjyauSfCzJ\n3yc5PskPJ3lkkq8leUiSN1XVj3T3vy15jRckeVqSLUnOXcTLSUmelOR9SQ5I8tdJ9kjy29saoqqO\nTnJ0kuyZvXbsTwgAXGerHSaf6e4/WHz8hap6XJIHVdVHkzwiyW26+yuLx19aVQ9O8vgkv7nkNY7p\n7rdv/aSqnpXkRd19/GLRl6rqd5OcWFW/0929cojuPi7JcUmyb93k+x4HAOax2mHyyRWffz3JDyY5\nJEkl+UxVLX18jyTvXvE1p6z4/NAkhy1iZKtdktwgyS2SfON6zgwArJLVDpNLV3zemSJil8XHP7KN\n51y44vPzV3y+S5I/TPKabXy/b123MQGAOax2mFyVf820xeQW3f2ea/m1H09yUHdv2fFjAQCraYgw\n6e4vVNVJSV5RVU/LFBs3SXJ4ktO6+3VX8+XPSfLmqjojyclJLktylySHdffTd+7kAMCONNJ1TH41\n05k5L0zyuSRvTvKjSc64ui/q7rcleWiSByb56OLPM5J85eq+DgAYz6ptMenuw7ex7DFLPr40yTGL\nP9v6+vdm2t2zrcfenuTt23oMAFg7RtpiAgBscMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYh\nTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYh\nTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABjGprkHANaQKy6fe4Kh9E/+59wjDOXMF/3Q\n3CMM5ZuH3XbuEcbylO17mi0mAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJADAMYQIADEOYAADDECYA\nwDCECQAwDGECAAxDmAAAwxAmAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJADAMYQIADEOYAADDECYA\nwDCECQAwDGECAAxDmAAAwxAmAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJADAMYQIADEOYAADDECYA\nwDA2zT3AHKrq6CRHJ8me2WvmaQCArTbkFpPuPq67N3f35t2yx9zjAAALGzJMAIAxCRMAYBjCBAAY\nxroNk6p6YlV9bu45AIDtt27DJMl+Se449xAAwPZbt2HS3cd0d809BwCw/dZtmAAAa48wAQCGIUwA\ngGEIEwBgGMIEABiGMAEAhiEzvCWfAAAG1klEQVRMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAY\nwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAY\nwgQAGIYwAQCGsWnuAQDWqisuvnjuEYZyx2ecOvcIQ/nHL35w7hGGsutTtu95tpgAAMMQJgDAMIQJ\nADAMYQIADEOYAADDECYAwDCECQAwDGECAAxDmAAAwxAmAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJ\nADAMYQIADEOYAADDECYAwDCECQAwDGECAAxDmAAAwxAmAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJ\nADAMYQIADEOYAADDWDNhUlW/XVWnzz0HALDzrJkwAQDWvx0SJlW1b1XdeEe81rX4njerqj1X83sC\nADvXdQ6Tqtq1qo6oqlclOTPJ3RfLb1RVx1XVWVV1blX9U1VtXvJ1j6mq86rqQVV1alWdX1XvqaoD\nV7z+06vqzMVzT0iyz4oRHpLkzMX3ut91/TkAgHFc6zCpqjtX1QuTfDXJq5Ocn+S/JXlfVVWStyS5\nVZKfSnLPJO9L8u6q2n/Jy+yR5JlJHpvkPklunOSvl3yPX0zy/yV5dpJDknw+yVNXjHJSkkcmuWGS\nd1TVlqr6g5WBcxU/w9FVdUpVnXJpLr62qwAA2Emqu6/5SVU3TXJkkqOS3DXJW5O8MsmbuvuiJc/7\nsSRvTHKz7r5wyfJPJHlVd7+wqh6T5PgkB3X35xePH5nk75Ls2d1dVR9K8unuftyS13hnktt19222\nMd++SX4+yaOT/NckH0hyQpKTu/u8q/vZ9q2b9L3qQde4DgC+T9XcEwxll732mnuEofzjFz849whD\n2XX/LR/r7s3X9Lzt3WLy/yY5NslFSe7Q3Q/r7tcsjZKFQ5PsleRbi10w51XVeUnukuSHlzzv4q1R\nsvD1JLsn+YHF5wcn+fCK1175+fd093e7+++6+4FJfiTJzZP8baZYAQDWiE3b+bzjklya5FeSnFpV\nr8+0xeRd3X35kuftkuSbmbZarPTdJR9ftuKxrZttrtMxL1W1R6ZdR4/KdOzJp5M8OckbrsvrAQDz\n2K4Q6O6vd/dzu/uOSR6c5Lwk/5Dka1X1kqq6x+KpH8+0teKK7t6y4s9Z12Kuzya594plyz6vyf2r\n6mWZDr798yRbkhza3Yd097Hdfc61+J4AwMyu9RaK7v5Id/9Gkv0z7eK5Q5J/qar/muSdST6Y5A1V\n9ZNVdWBV3aeq/nDx+PY6NslRVfW4qrp9VT0zyb1WPOdRSd6eZN8kj0jyQ939O9196rX9mQCAMWzv\nrpzv090XJ3ltktdW1Q8muXxx4OpDMp1R8/IkP5hp184HMx2Mur2v/eqqum2S52Y6ZuWNSf44yWOW\nPO1dSW7R3d/9/lcAANai7TorZz1zVg5wnTkrZxln5SznrJzldvRZOQAAO50wAQCGIUwAgGEIEwBg\nGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBg\nGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjABAIYhTACAYQgTAGAYwgQAGIYwAQCGsWnuAQDWrO65\nJxjKFeefP/cIQznilveYe4TBbNmuZ9liAgAMQ5gAAMMQJgDAMIQJADAMYQIADEOYAADDECYAwDCE\nCQAwDGECAAxDmAAAwxAmAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJADAMYQIADEOYAADDECYAwDCE\nCQAwDGECAAxDmAAAwxAmAMAwhAkAMAxhAgAMQ5gAAMMQJgDAMIQJADAMYQIADEOYAADDECYAwDCE\nCQAwDGECAAxj09wDzKGqjk5ydJLsmb1mngYA2GpDbjHp7uO6e3N3b94te8w9DgCwsCHDBAAYkzAB\nAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjAB\nAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGMIEABiGMAEAhiFMAIBhCBMAYBjCBAAYhjAB\nAIYhTACAYQgTAGAYwgQAGIYwAQCGIUwAgGEIEwBgGNXdc88wq6r6VpIz5p4jyX5Jzp57iIFYH8tZ\nH8tZH8tZH8tZH8uNsj5u3d03u6YnbfgwGUVVndLdm+eeYxTWx3LWx3LWx3LWx3LWx3JrbX3YlQMA\nDEOYAADDECbjOG7uAQZjfSxnfSxnfSxnfSxnfSy3ptaHY0wAgGHYYgIADEOYAADDECYAwDCECQAw\nDGECAAzj/wLFQzeyW1lQOAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "translate(u'hace mucho frio aqui.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 677
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 458390,
     "status": "ok",
     "timestamp": 1564762822189,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "zSx2iM36EZQZ",
    "outputId": "3e76fb22-6544-4a19-b631-f41f697fde87"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> esta es mi vida . <end>\n",
      "Predicted translation: this is my life . <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmoAAAJwCAYAAAA5n02CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xm4ZQdZ5/vfSyokQojIHJTJRhwY\nO5QM0g2ReKWllefKpbWVYABv0o+tLX1p9crTDy1NiwoGaWzUJoDMKpjbNiKDFww0NOMNaUQGZZ4M\nYR4SAklI3vvH3iWHk6qQOjWsd598Ps9zntpn7X32ec96qmp/z1p7rVXdHQAA5rnO0gMAALB/Qg0A\nYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihNlBVfVdVnVtVd156FgBg\nOUJtptOTnJLkkQvPAQAsqFyUfZaqqiQfTvKqJD+W5JbdfcWiQwEAi7BFbZ5TktwgyS8m+VqSBy46\nDQCwGKE2z+lJzunuS5L8yfpzAOBayK7PQarq+kk+keSfd/frq+puSd6U5KTu/sKy0wEAR5starP8\nH0k+092vT5LufnuS9yX5l4tOBQAbpKquX1U/U1XfuvQsh0qozfKwJC/YtuwFSR5+9EcBgI31E0me\nndXr6kaz63OIqrpVkg8l+d7uft+W5d+R1VGg39fd711oPADYGFX1miQ3T3JJd+9dep5DIdQAgF2j\nqm6b5L1J7pHkzUlO7u53LznTobDrc5CquvX6PGr7ve9ozwMAG+hhSV6/fp/3y7PhZ08QarN8KMlN\nty+sqhuv7wMArt7PJHn++vYLkzz0QBtBNoFQm6WS7G9f9AlJvnqUZwGAjVJVP5DkpCTnrBe9NMn1\nkvzQYkMdoj1LD0BSVb+7vtlJfrOqLtly9zFZ7Wd/+1EfDAA2y+lJXtLdFydJd19WVS/O6uwJr1py\nsJ0SajPcef1nJfneJJdtue+yJOcnOetoDwUAm6KqjsvqtBw/te2uFyT5y6o6YV/AbRJHfQ6x3n/+\n4iSP7O6Llp4HADZJVd0kq+tjv6C7r9x232lJXt3dFy4y3CEQakNU1TFZvQ/trpt8GDEAcPg4mGCI\n7r4iyUeSXHfpWQCAGWxRG6SqTs9q3/pp3f2ZpecBgOmq6kPZ/xkTrqK7v/MIj3PYOZhgll9Kcrsk\nf19VH0/y5a13dvddFpkKAOZ62pbbJyR5dJK3JnnTetm9szp7wpOP8lyHhVCb5Zxv/hAAYJ/u/ocA\nq6rnJHlid//G1sdU1WOS3PEoj3ZY2PUJAOwKVfWlrK7t+f5ty2+f5PzuPnGZyXbOwQQAwG7x5SSn\n7Gf5KUku2c/y8ez6HKSqrpvk32d1QMGtkxy79f7uPmaJuQBgQzwlye9V1d4kb14vu1dWVyx43FJD\nHQqhNst/SvKTSX4zq79sv5zktkn+ZZLHLjcWAMzX3U+qqg8neVRWVylIkvckOb27X7zYYIfAe9QG\nWR9i/HPd/cqquijJ3br7A1X1c0lO7e6HLDziSFX1iHx9K+Q3nIduEw/Fht2mqr4tyY9k//9GH7/I\nULAhbFGb5eZJ9l2V4OIkN1zffmWSJy4y0XBV9ctJHpPk6Unum+T3k9x+fdv1UWFhVXWvJC9LcmmS\nmyb5+yQnrT//cBKhxhFRVTfMtvfid/fnFhpnxxxMMMtHk9xyffv9SR6wvn3vJF9ZZKL5zkhyZnc/\nJsnlSZ7W3Q/K6nw5t1l0MiBJfjvJC5N8e1aXybt/VlvWzotfQDnMquo2VfWKqvpKks8m+fT64zPr\nPzeOLWqz/FmSU7N6A+RTk/xxVZ2R1X9wv73kYIN9R1YnNkxWMbvv0Os/Xi8/Y4mhgH9wlyQ/291d\nVVckOa67P1hV/3eSP8oq4uBweXZWe6N+NskFuYZXLJhMqA2y3iq07/Y5VfWxJPdJ8t7u/ovlJhvt\nwiQ3yWpr5Eey2vr49qx2f278P1DYBS7bcvuTWW3pfk9Wb++45X6/AnbuHknu1d3vXHqQw0WoDVJV\n903yxu7+WpJ091uSvKWq9lTVfbv7dctOONK5SR6U5Pwkz0rylKr6iSQnJ9nII3xglzk/yfcneW+S\n1yb59aq6eZLTkrxjwbnYnT6U5LilhzicHPU5yHq3wEnd/alty2+c5FPOo3ZVVXWdJNfZF7dV9ZNZ\nb4VM8vTuvnzJ+eDabn0+qxt092uq6qZJnpev/xt9RHf/zaIDsqtU1f2T/GqSf7396gSbSqgNUlVX\nJrl5d3962/I7JDlvEy99caRV1a2TfKy3/UWuqkpyq+7+6DKTAXC0rU9tdVySY7I6svhrW+/fxNdR\nuz4HqKo/X9/sJC+oqku33H1MkjsleeNRH2wzfCirQ/0/tW35jdb32QoJcO3xC0sPcLgJtRk+u/6z\nknw+33gqjsuS/M8kzzjaQ22Iyv4PGjghq1MBAEfZ+uTd12h3jZNSczh193OXnuFwE2oDdPcjkmR9\n2YuzuvvLy040X1X97vpmJ/nNqtp6sd1jsjry5+1HfTAgSZ625fYJSR6d1ely3rRedu+s/o0++SjP\nxbXA+mCVhyX5R0ke292fqar7JLmguz+07HQHz3vUBlm/MT7dfeX681sk+dEk7+5uuz63qKrXrG/e\nL6v//LeeAuCyrM54flZ3v+8ojwZsUVXPyeoUQ7+xbfljktyxu09bZDB2paq6e5K/yuqtL3dM8j3r\n8/Y9Lskduvunl5xvJ4TaIFX1iiSv7O6nVtUJSf42yfWz+o30Z7v7eYsOOFBVPTvJo7r7S0vPAlxV\nVX0pycnbj8CrqtsnOX8T39zNXOtf4l/X3b+2PrDgrutQu3eSP+nujbtijV2fs+xN8ivr2w9O8qUk\nt0vy0CS/lNVh7Wyxb7fxPlX1LVkd+v++7v7IMlNtHuvtwKrqwUle2t2Xr28fUHf/t6M01ib5cpJT\nsros3lanJLlk+4PhEN09q6sSbPeJrK6nvXGE2iwnJPnC+vYPJ/mz9YvDuUl+b7mx5lrvVnlrd/9+\nVV03q/fB3DHJZVX14939ikUHHMp6OyjnJLlFVkcWn3M1j+s4ynh/npLk99bnU3vzetm9kpye5HFL\nDcWu9ZUk37af5d+Tq54dYCO4KPssH01yn6q6flYXZH/VevmN4jfPA3lAvv6f/4OS3CCrF9XHxYvA\n1bHerqHuvs6+k1Cvbx/oQ6TtR3c/Kas3dt85ye+sP+6c5PTudlF2DreXJPm1qtp3dYKuqtsmeWKS\n/2epoQ6F96gNUlX/KqujpS7O6rqVJ3f3lVX1i0n+9+6+/6IDDlRVX01y++7+eFU9M8kXu/vfrf9h\n/k1332DRAYey3nZufUTZfZLcLN/4y2539x8sMxWQJFV1YpKXJ7lLVu/xvjCrXZ5vTPIjm3hWBbs+\nB+nup1fVeUluneRV+47+TPKBJI9dbrLRLkxyp6r6RFZbic5cLz8hictHHZj1tgNVdVqSZ+br5zzc\n+ptuJxFqsKD1gWX/ZH0pqZOz+mXq/O5+9bKT7ZxQG6KqvjXJXbr79Unetu3uLyR599GfaiP8YZIX\nJbkgyRVZHZadJPfM6qhZ9s9625knJHlSksfvu74sV7U+0vM71+evuihXc/JbR31yuGx9He3uc5Oc\nu+W++2R1qqvPLzbgDgm1Oa5M8oqqekB3v2Hfwqq6a1Z/2b59sckG6+7HV9U7k9wmyYu7e9/51L6W\n1XsS2A/rbcdOTPIckfZN/ZskF61v77pL+jDWrnwddTDBEN19UVZvgvyZbXc9LMlfdvdnjv5UG+Mr\nSX4oyauq6lbrZdfN6r1+HJj1dvBemOSfLz3EdN393O7ed83iH8/q79Qfr5d/w8eCY7LL7NbXUaE2\ny/OS/Iv16RL2Xangp5M8Z8mhJquqhyZ5cZL3ZnXOuWPXd10nXz8nHdtYbzv26CQ/UlX/var+U1X9\nh60fSw831CVJnpvkk1X1zKq639IDsavtutdRoTbLq7LayvGj689PzWoLx0sXm2i+X0lyRnf/X1nt\nttvnzUnutsxIG8F625l/leSfJfmBrLYU/YstHw9ZcK6x1pfsuXlWu0NvmdUW3I9U1W9V1Z2WnY5d\naNe9jgq1QdZHeb4gX99s+7AkL+puR+Ed2Hfl6xd63urirN5PxP5Zbzvz2CT/rrtv1t136u47b/m4\ny9LDTdXdX+7uF3T3A7N6n9BvZ/VC+vZlJ2O32Y2vow4mmOd5Sd5WVbfO6jf2UxeeZ7oLktwhq/PO\nbXXfrE5rwv5ZbztzTJI/X3qITVVVxye5f1anhLlDko8tOxG71K56HbVFbZjufleSd2b1puWPd/db\nFx5purOT/O760OskuVVVnZ7VKRSc0+rArLedeXZW197lGqqVH66q5yb5ZFZ/vy5Icmp3327Z6diN\ndtvrqC1qMz0vyX9O8u+XHmS67n7S+tw5r0pyfJLXJLk0yVnd7fqoB2C97dj1kvyfVfWAJO/ItpMD\nd/cvLjLVbJ/Ianf6K5I8PMnLtpwOhh2oqvck+a7u9hp+YLvmddQlpAaqqhtl9cbbp3f3hUvPswmq\n6npJvi+rrcTv7m6nmLgGrLeDU1WvuZq722Xerqqqzkjyp939haVn2S2q6heS3Li7/+PSs0y1m15H\nhRoAwFDeowYAMJRQAwAYSqgNVlVnLj3DJrLeDp51tjPW285YbwfPOtuZ3bDehNpsG/8XbCHW28Gz\nznbGetsZ6+3gWWc7s/HrTagBAAx1rT/q87p1XB+f6y89xn5dnktzbI5beoyNY70dPOtsZ6y3nbHe\nDp51tjOT19tF+fxnuvum3+xx1/qT5R2f6+eetdFXlwAANsyr+5ztl/DbL7s+AQCGEmoAAEMJNQCA\noYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFAD\nABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJ\nNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADDUy1KrqlKrqqrrJoTwGAGCTjQi1qnptVT3tIL/s\njUlOSvLZIzASAMDi9iw9wE5192VJLlx6DgCAI2XxLWpV9Zwk90vy8+tdmZ3ktuu771pVb6mqS6rq\nvKo6ecvXfcOuz6r61qp6flV9qqq+WlUfrKp/e7R/HgCAw2XxUEvyqCRvSvLsrHZlnpTkY+v7fjPJ\nryY5OatdnC+sqjrA8/x6kjsn+dEk353kkUn+/siNDQBwZC2+67O7v1hVlyW5pLsvTJKq+p713Y/t\n7teslz0+yf9M8u1JPr6fp7pNkvO7+63rzz9yoO9ZVWcmOTNJjs/1DsvPAQBwuE3YonZ13rHl9gXr\nP292gMf+QZKfrKq/rqqzqup+B3rS7j67u/d2995jc9zhmhUA4LCaHmqXb7nd6z/3O3N3vyKrrWpn\nJblJkpdV1bOP7HgAAEfOlFC7LMkxh/ok3f2Z7n5+dz88yc8mOb2qbDIDADbS4u9RW/twkntU1W2T\nXJwdBOT6PWznJ3lXVj/Xg5N8sLsvPWxTAgAcRVO2qJ2V1Va1dyf5dJJb7+A5Lk3yhCR/neQNSW6Q\n5McO14AAAEdbdfc3f9QudmLdqO9Zpy49BgBwLfLqPudt3b33mz1uyhY1AAC2EWoAAEMJNQCAoYQa\nAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhK\nqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCA\noYQaAMBQQg0AYCihBgAwlFADABhqz9IDLK2ue2z23OI7lh5j4/Tx1116hI3z8694+dIjbKTfv9/9\nlx5hI13xyU8tPcLG6St76RE205VXLD3BrmaLGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJq\nAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAo\noQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAA\nhhJqAABDCTUAgKGEGgDAUBsdalX1nKr6i6XnAAA4EvYsPcAhelSSWnoIAIAjYaNDrbu/uPQMAABH\nyq7Z9VlV962qN1fVxVX1xap6a1XdaekZAQB2aqO3qO1TVXuSvCTJs5I8NMmxSU5OcsWScwEAHIpd\nEWpJTkxywyQv7e4PrJf97YEeXFVnJjkzSY4/5gZHfjoAgB3Y6F2f+3T355I8J8lfVtXLqurRVXXr\nq3n82d29t7v3XveYbzlqcwIAHIxdEWpJ0t2PSHLPJK9L8qAkf1dVD1h2KgCAnds1oZYk3f3X3f3E\n7j4lyWuTnL7sRAAAO7crQq2qbldVv1VVP1BVt6mqH0xylyTvXno2AICd2i0HE1yS5A5J/jTJTZJ8\nMskLkzxxyaEAAA7FRodadz98y6cPXmoOAIAjYVfs+gQA2I2EGgDAUEINAGAooQYAMJRQAwAYSqgB\nAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGE\nGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAY\nSqgBAAwl1AAAhtqz9ABL68suz9f+/hNLj7F5rrxi6Qk2zu+f+r8tPcJGusfLP7D0CBvpjWfcfekR\nNs6eCz639Agb6YpPfnrpETbTZdfsYbaoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCU\nUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAA\nQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEG\nADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFDjQq2qXltVf1BVT66qz1XVp6vqUVV1XFX9\nXlV9oao+WlUPWz/+3Kp62rbnOLGqLqmqBy/zUwAAHLpxobb20CQXJblnkt9K8p+T/Pck702yN8lz\nkzyzqk5K8owkP11Vx235+p9KcnGSlx7NoQEADqepofau7n5cd78vye8k+UySy7v7qd39/iSPT1JJ\n7pPkvyW5MsmPb/n6RyZ5Xndfvr8nr6ozq+q8qjrv8lx6RH8QAICdmhpq79h3o7s7yaeS/M2WZZcn\n+XySm3X3pUmen1WcparumOQeSZ51oCfv7rO7e2937z02xx3oYQAAi9qz9AAHsH1LWB9g2b7QfGaS\nd1TVrbMKtjd193uO7IgAAEfW1C1qB6W735XkLUnOSHJakj9cdiIAgEM3dYvaTjwjyX/Nasvbixae\nBQDgkO2KLWprL0pyWZIXd/dFSw8DAHCoxm1R6+5T9rPsTvtZdotti26Y5FtyNQcRAABsknGhdrCq\n6tgkN07yG0n+V3e/YeGRAAAOi92w6/M+ST6R5AeyOpgAAGBX2Pgtat392qxOfgsAsKvshi1qAAC7\nklADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoA\nAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCih\nBgAwlFADABhKqAEADCXUAACG2rP0ACNcecXSE3At8LWPfGzpETbSG//xtyw9wka6wf/4xNIjbJx3\nv/IOS4+wkW7z5M8uPcKuZosaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQa\nAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhK\nqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCA\noYQaAMBQQg0AYKiRoVZVz6mqv9h+e/35darq6VX12arqqjplsUEBAI6gPUsPcA08Kklt+fyBSR6R\n5JQkH0zyuQVmAgA44saHWnd/cdui2yf5RHe/cYl5AACOlpG7Prfavhs0yVOS3Hq92/PD6+VVVb9S\nVR+oqq9U1d9U1WnLTQ0AcOjGb1Hb5lFJPpLkkUm+P8kV6+W/nuQhSX4+yd8luXeSZ1TV57v7ZUsM\nCgBwqDYq1Lr7i1V1UZIruvvCJKmq6yd5dJIf7u7Xrx/6oaq6R1bhdpVQq6ozk5yZJMfnekdldgCA\ng7VRoXYA35fk+CSvrKresvzYJB/e3xd099lJzk6SE+tGvb/HAAAsbTeE2r732f1Yko9uu+/yozwL\nAMBhsxtC7d1JLk1ym+4+d+lhAAAOl40Pte6+qKrOSnJWVVWS1yU5Icm9kly53s0JALBxNj7U1h6b\n5JNJfinJHyT5UpK3J3nSkkMBAByKkaHW3Q/f3+3152clOWvbsk7yX9YfAAC7wvgT3gIAXFsJNQCA\noYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFAD\nABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJ\nNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADLVn6QEArtaVVyw9wUa6+P5fWnqEjfNH733K0iNs\npEe/5V8vPcJmetU1e5gtagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAA\nQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEG\nADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYS\nagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGGrP0gMsoarOTHJmkhyf6y08DQDA/l0rt6h199nd\nvbe79x6b45YeBwBgv66VoQYAsAmEGgDAULs21KrqF6rqb5eeAwBgp3ZtqCW5SZLvXnoIAICd2rWh\n1t2P6+5aeg4AgJ3ataEGALDphBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AICh\nhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMA\nGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIbas/QAABx+\nffllS4+wcX71IY9ceoSNdO5Ln7X0CBvpmJOu2eNsUQMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoA\nwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqo\nAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AICh\nhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMtTGhVlW/VFUfXnoOAICjZWNCDQDg2uawhFpVnVhVNzwc\nz3UQ3/OmVXX80fyeAABH045DraqOqaoHVNUfJbkwyV3Xy7+1qs6uqk9V1UVV9T+qau+Wr3t4VV1c\nVadW1Tur6stV9Zqqut225/+Vqrpw/djnJTlh2wgPTHLh+nvdZ6c/BwDAVAcdalV1x6p6UpKPJXlR\nki8n+WdJXldVleRlSb49yY8m+cdJXpfk3Ko6acvTHJfkMUkemeTeSW6Y5L9u+R4/keTXk/xakpOT\n/F2SR28b5YVJfjrJDZK8qqreX1X/YXvwAQBsqmsUalV146r6xap6W5L/leR7kjwqyS26+4zufl13\nd5IfTHK3JA/p7rd29/u7+7FJPpjkYVueck+Sn18/5h1Jzkpyyjr0kuTfJnludz+9u9/b3U9I8tat\nM3X317r75d39U0lukeQ31t//fVX12qp6ZFVt3wq37+c5s6rOq6rzLs+l12QVAAAcddd0i9q/SfLU\nJF9NcofuflB3/2l3f3Xb4+6e5HpJPr3eZXlxVV2c5E5J/tGWx13a3X+35fMLklw3ybetP//eJG/a\n9tzbP/8H3f2l7v7D7v7BJN+f5OZJnpXkIQd4/Nndvbe79x6b467mxwYAWM6ea/i4s5NcnuRnkryz\nqv4syfOT/FV3X7HlcddJ8skk/3Q/z/GlLbe/tu2+3vL1B62qjstqV+tpWb137V1ZbZV7yU6eDwBg\ngmsURt19QXc/obu/O8kPJbk4yZ8k+XhVPbmq7rZ+6PlZbc26cr3bc+vHpw5irvckude2Zd/wea38\nk6p6elYHM/yXJO9PcvfuPrm7n9rdnz+I7wkAMMpBb8Hq7jd3988lOSmrXaJ3SPL/VdU/TfLqJG9I\n8pKq+pGqul1V3buq/uP6/mvqqUlOr6ozquq7quoxSe657TGnJfl/k5yY5KeS3Kq7f7m733mwPxMA\nwETXdNfnVXT3pUnOSXJOVd0syRXd3VX1wKyO2HxGkptltSv0DUmedxDP/aKq+s4kT8jqPW9/nuR3\nkjx8y8P+KquDGb501WcAANh8tTpY89rrxLpR37NOXXoMABZWd7/j0iNspFe+9IVLj7CRjjnp/W/r\n7r3f7HEuIQUAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgB\nAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGE\nGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUHuWHgAAJui3vWvpETbS\nA255t6VH2FDvv0aPskUNAGCRR2IpAAACNElEQVQooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUA\ngKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQ\nAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABD\nCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABD7Vl6gCVU1ZlJzkyS43O9hacB\nANi/a+UWte4+u7v3dvfeY3Pc0uMAAOzXtTLUAAA2gVADABhKqAEADCXUAACGEmoAAEMJNQCAoYQa\nAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhK\nqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCA\noYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAENVdy89w6Kq6tNJPrL0HAdwkySfWXqI\nDWS9HTzrbGest52x3g6edbYzk9fbbbr7pt/sQdf6UJusqs7r7r1Lz7FprLeDZ53tjPW2M9bbwbPO\ndmY3rDe7PgEAhhJqAABDCbXZzl56gA1lvR0862xnrLedsd4OnnW2Mxu/3rxHDQBgKFvUAACGEmoA\nAEMJNQCAoYQaAMBQQg0AYKj/HyzTthGxqa3hAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "translate(u'esta es mi vida.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 677
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 458790,
     "status": "ok",
     "timestamp": 1564762822599,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "A3LLCx3ZE0Ls",
    "outputId": "ef26532d-dcf7-4b63-ce8d-c144d247ff2e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> ¿ todavia estan en casa ? <end>\n",
      "Predicted translation: are you still at home ? <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmoAAAJwCAYAAAA5n02CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmcJXV97//3B4YlgKKiIi6ocYm7\nRieu0WiIgonxxuVqXEEjxC1uV5PrTYwao/5cE4zmKu77ft1iXIPGJfoziBoVN1RERURcQGQT5nP/\nqDOX7naQgWG6vj39fD4e/Zhz6pzu/nQ9Zvq8pupUVXV3AAAYz05zDwAAwJYJNQCAQQk1AIBBCTUA\ngEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJtQFU1bWq6siquuHcswAA4xBqYzg4ye2T\nPHjmOQCAgZSLss+rqirJcUk+lOSPk1yxu8+ddSiGUVVXSLLr0mXdffxM4wCwymxRm9/tk1wiyaOS\nnJPkD2edhtlV1d5V9eqqOiPJ95N8e8UHAOuEUJvfwUne1t2nJ3nT4j7r23OT3DjJnyQ5M8l9kzwh\nyfeS3HvGuQBYZXZ9zqiq9kzygyR/1N0fr6qbJPlUkv26+2fzTsdcqup7Se6z+DtxapKbdvexVXWf\nJA/u7jvOPCIAq8QWtXndI8nJ3f3xJOnuzyf5RpI/nXUq5napJN9Z3D4lyT6L259KcutZJgJY46pq\nz6p6YFXtPfcsF4ZQm9cDkrxuxbLXJTlk9UdhIN9M8puL219J8qeLg07unuQns00FsLbdK8krM732\nrhl2fc6kqq6S6Y3h1+3ubyxZfuVMR4Fer7u/PtN4zKiqHpvk3O5+QVX9fpJ/SbJLpv9YPbq7Xzjr\ngABrUFV9JMm+SU7v7o1zz7O1hBoMrqr2T7IxyTe6+4tzzwOw1lTV1ZJ8PcnNk3w603t/j5lzpq1l\n1+eMqmr/xS6tLT622vMwpu4+vrv/j0gDuMgekOTji/eC/2vW0BkWbFGbUVWdm+kIz5NWLN8nyUnd\nvfM8k7HaqupxSf65u89c3D5f3f38VRoLYIdQVd9I8vTuflVV3SPJ4Umu0msggoTajKpqU5J9u/tH\nK5ZfNckx3b3nPJOx2qrq20k2dvePF7fPT3f3b/6axwFYoqpuneSDSa7Q3adV1a5JTkxy7+7+0LzT\nXbANcw+wHlXVCxY3O8kzq+r0JQ/vnGkf+udXfTBm091X39JtALbZwUne1d2nJUl3n11Vb8l0hgWh\nxhbdcPFnJblukrOXPHZ2kqMznZ2edaiqbrJ4HwUA26Cqdst0Wo77rHjodUk+UFV7bQ64Udn1OZPF\nQQRvyXSm+Z/PPQ/jWOwSPybJa5O8obu/O/NIAGtSVV020zW0X9fdm1Y8dv8kH+7uE2cZbisJtZlU\n1c6ZruN447VyiDCro6quneR+mf4H+JtJPpEp2t7W3afMOdtcqmr3JI9OckCSy2fFEevdfaM55gLY\n3oTajKrq2CT3tJuL81NVt8gUbfdKcskk7+3u/z7vVKuvql6R5G5J3prkhEzv7/x/uvupc8wFsL0J\ntRlV1cGZtprcv7tPnnsexrUIthcnudF6PG1LVf0kyb26+8NzzwKMb3H0/FYFzuhH0juYYF6PT3L1\nJN+vqu8l+cXSB+3OWd+q6uqZtqbdL8k1k3wsyUNmHWo+pyfxXj1gay291N5eSR6X5DNJPrVYdqtM\nZ1h43irPdaHZojajqnryr3vc7pz1qaoekSnObpHkS5mOTnpDd39/1sFmVFWPSnL9JA9dCyeoBMZR\nVa9K8vXufsaK5U9Mcv3uvv8sg20loQaDqarjk7wx01FKLhuVpKrek+S2SU7JdETsL5c+3t13nWMu\nYHxVdWqma3seu2L5NZMc3d2XnGeyrWPXJ4znqrYa/YqTk7xj7iGANekXSW6f5NgVy2+f6W0VQxNq\nM1pcxuKvMx1QsH+SXZY+vh7fNM50jagkqaorZvp7seuKxz82x1xz6u4HzT0D4/K7lAvwD0leVFUb\nk3x6seyWma5Y8JS5htpaQm1eT0ty7yTPzPQX6QlJrpbkT5M8ab6xmNMi0N6YaVdfZ7qCxdItbF50\nYDm/Szlf3f3sqjou07kY77VY/JUkB3f3W2YbbCt5j9qMFocPP6y7319VP09yk+7+ZlU9LMkB3X3P\nmUdkBotr0O2T5BFJ/jPJQUn2TfJ3SR67Fi4ivD1U1YNy3haTlVsZhz68nu3L71J2ZDtd8FPYjvbN\n9MboJDktyaUWt9+f5E6zTMQIfi/JX3X3VzNtSftRd/+fJH+VacvBulNVT8h0GP1nM20peWemI2Iv\nk+QV803GIPwuZatU1aWq6jJLP+ae6YIItXkdn+SKi9vHJjlwcftWSc6YZSJG8BuZ3jyfJD/JdMmk\nZHohWq/n1js0yWHd/cRMR3y+cHGk5/OSXHXWyRiB36Wcr6q6alW9r6rOSPLjJD9afJy8+HNo3qM2\nr3dkunbhp5McnuSNVXVokislec6cgzGrrya5TpLjknw+yUOr6ruZdoWu13OpXTnTySqT6YV38+H0\nb1wsP3SOoRiG36X8Oq/MtJX1z7KFS9CNznvUBrK4TNBtMp2Y71/mnod5VNX9kuzS3a+qqptm2n2z\nT5KzMr359a2zDjiDqvpWpuviHl1V/5nkFd39v6vqoCSv7+59Zh6RgVTVLZPcOn6XkqSqTktyy+7+\n0tyzXBRCbUZVdbsk/9Hd56xYviHJrdfjaRj4VVW1R6YtbMev12vCVtXLknyvu59SVQ/NdGTfp5Pc\nNMlbutsWNWCLquqLSQ7p7s/OPctFIdRmVFXnJtmvu09asXyfJCc59w9MqmqnJDtt/k9NVd07i63P\nSV7S3b/8dZ/Pjq2q7pXkZ939wcX9v01yWJIvZ3qB/sGc8zGvqvr9JP8zycNXXp1gLRBqM6qqTUn2\n7e4frVh+7SRHjX5ZCy4+VbXVRy5294O35ywjqqr9k3x35RUbqqqSXKW7j59nMkZQVcckeUx3f3Dx\ndoH/SPK3mU5tc2J333fWAZnV4pQtu2U6B+VZSZbtxRr9tdbBBDOoqncvbnaS11XVWUse3jnJDTL9\nomH9uNyK+7dLsinJ5mt93iDTUdrrdXf4t5Psl+SkFcsvs3jM1uf17apJvra4fbck71yc5PSDST4w\n31gM4pFzD7AthNo8frz4s5L8NMsPHz87ySeSvHS1h2I+3f3Hm29X1RMz/Z14UHf/YrFszyQvz3nh\ntt6svDrDZnslOXOVZ2E8Zya5xOL2ATnv3HqnLFnOOtXdr557hm1h1+eMqurJSZ67+cUYkqSqfpDp\nbOrHrFh+/ST/1t1XmGey1VdVL1jcfESmQ+yXXkB55yQ3T3J2d99mtWdjHFX1zkznH/xEpktGXa27\nT6iqA5O8oLt/a9YBmV1V7ZvkAUmukeRJ3X1yVd0myQnd/e15p/v1nPB2Xk/Lkq1pVXWFqnpIVd16\nxpmY31457+SdS+2XZI9VnmVuN1x8VJLrLrl/wyTXTHJ0kkPmGo5hPDLT3oh7Jnlod5+wWH7n2PW5\n7lXVzTLtGr9fpnOpbX5P2h2TPH2uubaWLWozqqr3JXl/dx9eVXtlOtHpnpleqP+su18z64DMoqpe\nlWn3zRMynYIiSW6Z5FlJPtLdh8wz2Xyq6pVJHt3dp849yygWp225SaYrVyz7T/fikmNAkqr6SJKP\ndfeTFwcW3Li7v1VVt0rypu4e+uomQm1GVfWjJL/f3V+sqgdmOnz4xpmq/3HdvV4vF7SuVdVvZLo0\n0oOT7LJYfE6m96g9vrtPP7/PXS8W6+g2Sb7R3d+Ze57VVlV/kOmqDFs60W87tQ+cp6pOTXKTRZwt\nDbWrJflqd+8+64AXwK7Pee2V5GeL23dK8o7F+aCOzLQfnXWou8/o7odnehH+7cXHZbr74es10qrq\nVVX18MXtXTNdNuqDSb5WVXeedbh5HJ7kvUmu3N07rfhYd5FWVbtW1VOr6utVdWZVnbv0Y+75mN0Z\nSS69heXXya8eST4coTav45PcZnFE34FJPrRYfpksf9M069O5mU7Rce7iYz07MOftBr5rpiP5rpDk\nKYuP9eZqSZ625L1Y693TkhycaUv0pkxvG3hRpiPsHz7jXIzhXUmeXFW7Le73Ymvas5K8fa6htpZQ\nm9fzk7w2yfcyXWx78zmybpf1exqGda+qNlTVczKduuULmf4u/LSqnl1Vu/z6z95hXTrn/c/3oCRv\nX1zR401JrjfbVPP5ZBJHMp7nXpkOInhJpv/UvKu7H5XkyZneMM769vhMG0B+lOmArE8kOTbT6Vv+\nZsa5torzqM2ou19SVUcl2T/Jh7p70+Khb2Y6xJz16dlJ7pPkoZl+oSTJbZM8M9N/rh4/01xzOjHJ\nDRanLjkw0+WBkuntA+vx8lEvTvLcqrpippBftg66++hZpprPvkk2n87mtCSXWtx+f6atJqxji4OQ\nfndxKambZvo9enR3f3jeybaOUJtJVe2d5Ebd/fEkKy8U+7Oc90uH9ee+SR7c3f+6ZNk3FwefvCzr\nM9RekeTNSU7ItMXk3xbLb5HpaOn15m2LP4/YwmOd9XelhuMzndLm+ExbSg7M9Hv1Vll+QnHWmaWv\ntd19ZKb3gG9+7DZJjunun8424FYQavPZlOR9VXVgd39y88KqunGmv0hXmm0y5rZ3pq2qK30z520p\nWFe6+++q6kuZLhX0lu4+e/HQOVmfW0yuPvcAg3lHplPafDrTgRZvrKpDM/0efc6cgzG7Nf9a6z1q\nM+nun2d6g+MDVzz0gCQf6O6TV38qBvGFJI/awvJHJ/n8Ks8ykjOS/EGSD1XVVRbLds20q2tdWZyS\n5HqZ3jD/viSbFsvumOlEwOtKdz+xu5++uP22JL+b5J+S3L27/3rW4ZjVjvBaK9Tm9Zok/31xuoFU\n1U6Zdnu9as6hmN1fJjm4qr5WVa9efHwtyf0zHc227lTV/ZK8JcnXM21N2nxQxU6Z1te6smR9fCPL\n18fOWZ/r4+lV9dDN97v7/+/u5ye5clU9bcbRGMOafq0VavP6UKatBHdZ3D8g0xaC98w20cAW/7jW\ng+OSXDvT+5D2Wny8NdNRfsfPN9as/jLJod392Ey7Ozf7dKaz86831sdyD0jyuS0s/2x+dUvKDq2q\n7lJVj6mqdXNN4K2wpl9r18sL35AWR3m+Luf9InlAkjcvTnrLCkuOit3RfTvJOd391919j8XH3yQ5\na/HYenStJJ/awvLTct51+9YT62O5y2c69cJKP850ROi6UFX/M9P79Z6Q5AtVdcOZRxrCWn+tFWrz\ne02Sg6pq/yR3S/LqmeeZTVV9pKpeWVWXXtx+d1UdPPdcM6hMR+6ttFeSM1d5llGckGkr40q3y5YP\nvNjRWR/LHZ/pFDYr3S7TeSrXi4dnuk70lTIdVPGhqrpTVe2/OD/jfovXmvVozb7WOupzZt395cXR\nbK9P8r3u/szcM83oS5nOl/XLxe1LJHlRVd1scfLKHVpVvWBxs5M8s6qWXp1i5yQ3z/o9mOCIJC+o\nqocs7l+lqm6b6ZxzT5ltqvlYH8u9JMk/LN6DtPn0CwdkOvfgejoq+DJZnDi9u5+xeLvI+xaP/U6m\n15lrZ/2dvmVNv9YKtTG8Jsk/JlnXRyd1918sufsXSVJV/5Tk/YvLfbytu18zw2irZfNuikpy3SRn\nL3ns7CRHJ3nuag81gu5+9uJ8SB9KsnuSj2TaFfzc7n7RrMPNwPpYrrufV1WXTfKCTO89SqZ/M4d3\n97Pnm2zVfT3T0cDHJUl3/31VvTzJfkm+kmnX3x6zTTe/NflaW91b2sPCaqqqy2QKk5d094lzzzOa\nqrp2kpcmuVl37zX3PNtbVb0yyaMXZ9NmiaraI9ML0U6ZTlS57k7NsZT1sdziusmbLyn2lfW2Pqrq\nkUnu0N33mHuWEa3V11qhBgAwKAcTAAAMSqgBAAxKqA2iqg6be4aRWB/LWR/LWR/LWR/LWR/LWR/L\nrbX1IdTGsab+4qwC62M562M562M562M562M562O5NbU+hBoAwKDW/VGfu9ZuvXv2nHuM/DJnZZfs\nNvcYw7A+lrM+lrM+lrM+lrM+lrM+lhtlffw8Pz25uy93Qc9b9ye83T175hZ1wNxjALCjqZp7Agb2\n4U1v/c7WPM+uTwCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCA\nQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJ\nNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBrflQq6pd5p4BAGB7GC7Uquqg\nqvp4Vf20qn5SVR+oqusuHrtaVXVV3aeqjqyqM5L8+eKxW1fVv1fV6VX1/ar631V1yVl/GACAbTBc\nqCXZM8k/Jrl5ktsnOSXJe6pq1yXPeWaSf05yvSTvrKobJvlgkncnuXGSuye5SZJXrN7YAAAXrw1z\nD7BSd7996f2qelCSUzOF2/cWi/+pu9+25DnPSPLm7n7ekmUPS/K5qrp8d5+04mseluSwJNk9e2yX\nnwMAYFsNt0Wtqq5RVW+oqm9W1alJfphpzv2XPO2oFZ92syT3r6rTNn8k+eTisWus/B7dfUR3b+zu\njbtkt+3xYwAAbLPhtqgl+ZdMW87+PMn3k5yT5JgkS3d9/mLF5+yU5GVJ/mELX+/722FGAIDtbqhQ\nq6p9klwnycO7+yOLZTfNBc95dJLrd/ex23lEAIBVM9quz58mOTnJoVV1zar6vSQvzrRV7dd5VpKb\nV9WLq+q3F597l6p6yfYeGABgexkq1Lp7U5J7J7lRki8leVGSJyU56wI+77+S3C7J1ZL8e5IvZDoy\n9IfbcVwAgO1qqF2fSdLdRya5wYrFey25XefzeUclOWh7zQUAsNqG2qIGAMB5hBoAwKCEGgDAoIQa\nAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDA\noIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKA2\nzD3A3GrnnbPz3peee4xhbDr99LlHGMpP7nPTuUcYyo9v1HOPMJTrHP79uUcYSp951twjjGWTfy9L\n1c62DS3zg617mrUGADAooQYAMCihBgAwKKEGADAooQYAMCihBgAwKKEGADAooQYAMCihBgAwKKEG\nADAooQYAMCihBgAwKKEGADAooQYAMCihBgAwKKEGADAooQYAMCihBgAwKKEGADAooQYAMCihBgAw\nKKEGADAooQYAMCihBgAwKKEGADAooQYAMCihBgAwKKEGADAooQYAMCihBgAwKKEGADAooQYAMKjZ\nQ62qHlhVP66q3VYsf31VvXtx+8+r6tiqOnvx56ErnttVdc8Vy46rqsdv/58AAGD7mD3Ukrw10xz/\nbfOCqto7yd2SvLyq7pbkhUn+MckNkhye5J+r6o9nmBUAYNVsmHuA7j6jql6f5MFJ3rJYfN8kpyZ5\nb5J/T/La7n7h4rGvV9XNkvxVkvdclO9ZVYclOSxJdt9pr22YHgBg+xlhi1qSvDTJHavqyov7D07y\n6u4+J8l1k3xyxfM/keR6F/WbdfcR3b2xuzfuWrtf1C8DALBdDRFq3f2FJEcnOaSqbpBkY5JXXNCn\nrbhdKx7f5eKbEABg9Q0RagsvTXJIkock+WR3f22x/CtJbrPiub+b5Jgl93+UZL/Nd6pq36X3AQDW\notnfo7bEG5M8P8nDkjx0yfLnJHlrVX02yQeTHJTkfknuvuQ5RyZ5RFX9R5JzkzwjyZmrMTQAwPYy\nzBa17v55poMJzsp5BxWku9+Z5C+SPDbTVrRHJ3l4dy89kOB/JPlWko8meVuSlyU5aVUGBwDYTkba\nopZMuyvf3N2/WLqwu1+c5MXn90ndfUKSO69Y/PaLfzwAgNUzRKhV1aWT3DbJnZLceOZxAACGMESo\nJflckssk+V/d/aW5hwEAGMEQodbdV5t7BgCA0QxzMAEAAMsJNQCAQQk1AIBBCTUAgEEJNQCAQQk1\nAIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCA\nQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQW2Ye4C59bnn5tyf/WzuMYZRG3aZe4ShnLxx\n09wjDOUSx+489whDOfeyl5x7hKEcf+e95x5hKPs/57NzjzCUPuusuUdYk2xRAwAYlFADABiUUAMA\nGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiU\nUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFAD\nABiUUAMAGJRQAwAYlFADABjUmg21qvpoVb1wa+8DAKw1G+Ye4IJU1SFJXtjde6146O5Jfrn6EwEA\nrI7hQ+38dPdP5p4BAGB7GmbXZ1Xdrqo+XVWnVdUpVfWZqnpkklcm2bOqevHxlMXz7doEAHZoQ2xR\nq6oNSd6V5OVJ7pdklyQ3TfLlJI9J8owk11g8/bQ5ZgQAWG1DhFqSSya5VJL3dPc3F8u+miRV9dtJ\nurtPvLi+WVUdluSwJNk9e1xcXxYA4GI1xK7PxfvNXpXkA1X13qp6XFXtvx2/3xHdvbG7N+6S3bbX\ntwEA2CZDhFqSdPeDktwiyceS3DXJ16rqwHmnAgCYzzChliTd/YXuflZ33z7JR5McnOTsJDvPORcA\nwByGCLWqunpV/X9VdeuqumpV3SHJjZIck+S4JLtX1R2r6rJV5U1lAMC6MMrBBKcnuXaStya5bJIf\nJnl9kmd19y+r6sVJ3phknyRPTfKUmeYEAFg1Q4Rad/8w05UGzu/xhyV52Iplt78w9wEA1pohdn0C\nAPCrhBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDA\noIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCE\nGgDAoIQaAMCghBoAwKA2zD3AEEqvbla77jL3CEO51qOOmnuEoex86b3nHmEoX33qteYeYSjPudPr\n5h5hKC991nXmHoEdgEIBABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAY\nlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQ\nAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAY1A4XalV1tarq\nqto49ywAANtihws1AIAdxZoMtao6qKo+XlU/raqfVNUHquq6i4e/vfjzPxdb1j4605gAANtkTYZa\nkj2T/GOSmye5fZJTkrynqnZdLEuSg5Lsl+TucwwIALCtNsw9wEXR3W9fer+qHpTk1EyR9r3F4h93\n94lb+vyqOizJYUmye/bYjpMCAFx0a3KLWlVdo6reUFXfrKpTk/ww08+y/9Z8fncf0d0bu3vjLtlt\nu84KAHBRrcktakn+JdOWsz9P8v0k5yQ5Jsmucw4FAHBxWnOhVlX7JLlOkod390cWy26a836Wsxd/\n7jzDeAAAF5s1F2pJfprk5CSHVtV3k1wpyXMybVVLkpOSnJHkwKo6LsmZ3X3KHIMCAGyLNfcete7e\nlOTeSW6U5EtJXpTkSUnOWjx+TpJHJXlIkhOSvGueSQEAts1a3KKW7j4yyQ1WLN5ryeMvS/KyVR0K\nAOBitua2qAEArBdCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBC\nDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0A\nYFBCDQBgUEINAGBQQg0AYFBCDQBgUBvmHmAIm86de4Jh7HSpveceYSibTj997hGGcu6PfzL3CEO5\n7nN/MPcIQ/n7q/zh3CMMpd7ec48wlH3v9Z25RxjLGVv3NFvUAAAGJdQAAAYl1AAABiXUAAAGJdQA\nAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAG\nJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXU\nAAAGJdQAAAZ1sYZaVX20ql54cX5NAID1yhY1AIBBCTUAgEFtj1DbqaqeUVUnV9VJVfXcqtopSarq\n0lX16qr6aVWdUVUfrqrrb/7Eqjqkqk6rqjtX1Ver6vSqendV7V1V96yqb1TVKVX12qr6jSWfV1X1\nl1X1zcXX/WJV3X87/GwAAKtme4Ta/ZKck+TWSR6Z5DFJ7r147FVJbpHkvyW5eZLTk7x/aXQl2S3J\n/1h8nQOSbEzy9iQHJ7lHkj9JcpckD1/yOX+f5M+SPCLJ9ZI8M8lLquqPtjRgVR1WVUdV1VG/zFnb\n+OMCAGwfG7bD1zymu/92cfvrVXVokgOq6qgkd03ye939sSSpqgckOT5TlL1syUyP6O6vLZ7zhiSP\nTbJvd5+8WPauJHdI8ryq2jPJ45Lcqbs/vvga366qm2cKt/euHLC7j0hyRJJcsi7TF+tPDwBwMdke\nofZfK+6fkOTySa6bZFOST21+oLtPqaovZtoKttlZmyNt4YdJTtwcaUuWbf6c6yXZPdOWuaXRtUuS\n47bh5wAAmNX2CLVfrrjfueBdrEsD65wtPPbrvubmP/8409a5XzcLAMCasT1C7fx8JVNU3SrJ5l2f\nl0xywySv3Iave0ySs5JctbuP3NYhAQBGsWqh1t3fWLy37CVVdViSnyV5epJTk7xhG77uz6vquUme\nW1WVKQL3SnLLJJsW70cDAFiqPlR5AAAKDklEQVRzVvs8ag9K8pkk7178uUeSg7r7jG38uk9K8pQk\nj0/y5SQfynSE6Le38esCAMymutf3QY+XrMv0LeqAuccYxoYrXXHuEYZyzgk/mHuEsazz3xcrbbja\n/nOPMJQTDt9j7hGGsvz4Nva913fmHmEoHzzjdZ/t7o0X9DxXJgAAGJRQAwAYlFADABiUUAMAGJRQ\nAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMA\nGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAYlFADABiUUAMAGJRQAwAY1Ia5B2As\nJx9w1blHGMo+7/7F3CMM5dxTTp17hKFsOvGkuUcYyplnX2vuEYZy9nf3nHuEoVzurK/PPcKaZIsa\nAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDA\noIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCE\nGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoHaoUKuqR1bV56rqF1X13ap64twzAQBc\nVBvmHuBidkCSv03y5SS3S/Kyqvpyd7973rEAAC68HSrUuvtuS+5+q6qekeSac80DALAtdqhQW6qq\n/leSXZK8aQuPHZbksCTZPXus8mQAAFtnh3qP2mZV9TdJHpPkjt19wsrHu/uI7t7Y3Rt3yW6rPyAA\nwFbY4baoVdUVk/xdkj/q7s/PPQ8AwEW1I25R2y9JJfnK3IMAAGyLHTHUvpLkd5L8yi5PAIC1ZEcM\ntRskeV2Sy809CADAttgRQ22PJL+V6YhPAIA1a4c7mKC7P5rpPWoAAGvajrhFDQBghyDUAAAGJdQA\nAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAG\nJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXUAAAGJdQAAAYl1AAABiXU\nAAAGtWHuARjLJY4/a+4RhnLuqafNPcJQaued5x5hKJvO8u9lqas/9PtzjzCUf/3ikXOPMJQ/euGf\nzD3CWL65dU+zRQ0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0A\nYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQ\nQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUGsm1Krq8VV13NxzAACsljUT\nagAA683FEmpVdcmqutTF8bUuxPe8XFXtvprfEwBgNV3kUKuqnavqwKp6Q5ITk9x4sXzvqjqiqk6q\nqp9X1b9X1cYln3dIVZ1WVQdU1Zeq6hdV9ZGquvqKr/+XVXXi4rmvSbLXihH+MMmJi+91m4v6cwAA\njOpCh1pVXb+qnp3ku0nenOQXSQ5K8rGqqiTvTXKlJHdJ8ttJPpbkyKrab8mX2S3JE5M8OMmtklwq\nyYuXfI97Jfn7JE9OctMkX0vyuBWjvD7JfZNcIsmHqurYqvrblcF3Pj/DYVV1VFUd9cucdWFXAQDA\nqtiqUKuqfarqUVX12SSfS3KdJI9OcoXuPrS7P9bdneQOSW6S5J7d/ZnuPra7n5TkW0kesORLbkjy\niMVz/ivJc5PcfhF6SfKYJK/u7pd099e7++lJPrN0pu4+p7v/tbvvk+QKSZ6x+P7fqKqPVtWDq2rl\nVrjNn3tEd2/s7o27ZLetWQUAAKtua7eo/UWSw5OcmeTa3X3X7n5rd5+54nk3S7JHkh8tdlmeVlWn\nJblBkmssed5Z3f21JfdPSLJrkksv7l83yadWfO2V9/+f7j61u1/R3XdI8jtJ9k3y8iT33MqfDwBg\nOBu28nlHJPllkgcm+VJVvSPJa5P8W3efu+R5OyX5YZLbbuFrnLrk9jkrHusln3+hVdVumXa13j/T\ne9e+nGmr3LsuytcDABjBVoVRd5/Q3U/v7t9K8gdJTkvypiTfq6rnVdVNFk89OtPWrE2L3Z5LP066\nEHN9JcktVyxbdr8mv1tVL8l0MMM/JTk2yc26+6bdfXh3//RCfE8AgKFc6C1Y3f3p7n5Ykv0y7RK9\ndpL/rKrbJvlwkk8meVdV3bmqrl5Vt6qqpy4e31qHJzm4qg6tqmtV1ROT3GLFc+6f5INJLpnkPkmu\n0t1P6O4vXdifCQBgRFu76/NXdPdZSd6W5G1Vdfkk53Z3V9UfZjpi86VJLp9pV+gnk7zmQnztN1fV\nbyZ5eqb3vL07yfOTHLLkaf+W6WCGU3/1KwAArH0XOdSWWrpbs7t/numI0Eefz3NfleRVK5Z9NEmt\nWPbMJM9c8elPWfL4CRd9YgCA8bmEFADAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQa\nAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDA\noIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCgNsw9AGPZ+aNHzz0CA+tNc0/AyM79\n8U/mHmEoB17xJnOPMJjj5h5gTbJFDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEIN\nAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBg\nUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBC\nDQBgUEINAGBQQg0AYFAb5h5gDlV1WJLDkmT37DHzNAAAW7Yut6h19xHdvbG7N+6S3eYeBwBgi9Zl\nqAEArAVCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEIN\nAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBg\nUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUNXdc88wq6r6UZLv\nzD1HkssmOXnuIQZifSxnfSxnfSxnfSxnfSxnfSw3yvq4andf7oKetO5DbRRVdVR3b5x7jlFYH8tZ\nH8tZH8tZH8tZH8tZH8uttfVh1ycAwKCEGgDAoITaOI6Ye4DBWB/LWR/LWR/LWR/LWR/LWR/Lran1\n4T1qAACDskUNAGBQQg0AYFBCDQBgUEINAGBQQg0AYFD/F04a3TExTMjUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "translate(u'¿todavia estan en casa?')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 701
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 459670,
     "status": "ok",
     "timestamp": 1564762823492,
     "user": {
      "displayName": "Will Chen",
      "photoUrl": "",
      "userId": "01179718990779759737"
     },
     "user_tz": -480
    },
    "id": "DUQVLVqUE1YW",
    "outputId": "9ab39734-7a3d-4555-abd2-62804f0b3be4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> trata de averiguarlo . <end>\n",
      "Predicted translation: try to figure it out . <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeIAAAKICAYAAACsbvkCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmUZAV59/HvT2YAAfcFBxNEcI27\njguvCqhR45KcRD1GExckEY2SkGM0xiWvK8EFNSZoIlEhRI0aY17cYtxAxbhk3CJuCAaNyqoIDAgD\nw/P+cWukuxmGnpmeem51fz/nzJnq29XVT9eZqW/fW3dJVSFJknpcp3sASZJWMkMsSVIjQyxJUiND\nLElSI0MsSVIjQyxJUiNDLElSI0MsSVKjVd0DSNJiJdkVuA1QwOlVdWnzSNJ2c41Y0uglWZXktcD5\nwNeBbwDnJ3lNktW900nbxzViSbPgNcATgWcCJ0+WPRA4kmGF4rlNc0nbLZ5rWtLYJTkLOKSqPrJg\n+aOAt1bVmp7JpO3npmlJs+AGwOmbWX46cMMpzyItKUMsaRZ8HfiTzSw/HPjalGeRlpSbpiWNXpID\ngI8APwa+MFl8P2Av4BFVdfI1fa00doZY0kxIshfwbOAOk0XfBt5cVT/pm0rafoZYkqRGHr4kaZSS\n3HOx962qr+zIWaQdyTViSaOU5EqGM2jlWu5aVbXTFEaSdgjXiCWN1a27B5CmwTViSaM2OYXlEcCb\nquoH3fNIS80QSxq9JOuBO1fVGd2zSEvNE3pImgX/ATy4ewhpR/A9Ykmz4JPAXyW5K/Bl4OK5n6yq\n97dMJS0BN01LGr3JHtTXxL2mNdMMsSRJjXyPWJKkRr5HLGkmJLkR8Ahgb2DnuZ+rqpe3DCUtATdN\nSxq9JPcDPgxcBtyM4SpMayYfn1FVd20cT9oubpqWNAteC7wTuCVwKcOhTHsD64BXN84lbTfXiCWN\nXpILgHtX1alJfg7sX1XfTnJv4F1VddvmEaVt5hqxpFmwYc7ts4FbTW6vB/aa/jjS0nFnLUmz4CvA\nvYFTgZOAVybZE3gS8N+Nc0nbzU3TkkYvyVrgelV1YpKbAccD92cI89Oq6hutA0rbwRCPQJLbAm8B\nDvcFRZJWFt8jHoenAgcBhzTPIUmaMteImyUJcAbwceA3gb2qamPrUNLIJPkGcI0vVh5HrFnmzlr9\nDgKuB/wJw1mDHgl8sHMgaYTet+Dj1cDdGd4nftP0x5GWjmvEzZIcB2yoqkOTvA64VVU9rnksaSYk\neR7D/5nDumeRtpUhbpRkd+BM4FFV9dkkdwc+D6ypqp/3TieNX5L9gHVVdaPuWaRt5c5avR4LnFdV\nnwWoqq8B3wOe0DqVNDsOAC7pHkLjkGT3JE9JcoPuWbaG7xH3ejLwjgXL3gEcDPz91KeRRirJBxYu\nYrjowz2Al01/Io3U44G3AocDRzfPsmhumm6S5FeB/wHuWFXfm7P8Vxj2ov61qjq1aTxpVJIcu2DR\nlcC5wKeq6mMNI2mEkpwI7AlcUlVru+dZLEMsSZp5SfZhONPafYAvAPesqm91zrRYvkfcKMnek+OI\nN/u5ac8jSTPsycBnJ/vafIThREkzwTXiRkk2Muwhfc6C5TcBzqmqnXomk8Ylyf+w+RN6FMP1iU8D\n3lZVC99L1gqR5HvAEVV1XJLHAm8EfrVmIHKuEfcKm39x2YPhxUXS4FjgxgxHFbxj8ud7k2UfADYC\n70/yu20Tqk2S/8Ow896mE798ENgN+PW2obaCe003SPI3k5sFHJlk7uEXOzG8x/G1qQ8mjde+wKuq\n6lVzFyb5c4YdGx+T5IXAXwDv6RhQrZ4KnFBV6wGqakOS9zIcgfLxzsEWw03TDSZ79gEcyHACj7kX\nPd/AsNf0UXP3ppZWsiQXMux8c9qC5bcBvlJV109ye+DLVbVHy5BqkWQX4CzgiVX10TnLHwD8B7Dn\npkCPlWvEDarqQZOdtN4LHFJVF3XPJI3cJcADGd4LnuuBXHVCj52AX0xzKI3C9RiOG553GFtVnZzk\nGQxv9Y06xK4RN0myE8P7wHeblV3spS5JXgD8X+DtwH9NFt+bYdPjK6rqVUmeAzyiqh7aM6W0bQxx\noySnAY+b7G4vaQuSPIHhKmV3mCz6DvDGqnrP5PPXBaqq3NFRM8UQN0ryVOCJwJOq6rzueSRpVmzh\nkLarqap9d/A428X3iHs9F7g18OMkPwIunvtJL3YuSddo7rmk9wCeA3yJYQdYgP0ZjkB53ZTn2mqG\nuNfCi51LmpjsKb1vVZ2X5CK2sPZTVdef3mQag6r6ZWAn13V/dVX91dz7TPYtuNOUR9tqbprWKCR5\nEMNm+r2Bned+rqoe3DKUWk3eunl3VV02uX2NquofpzSWRmgxh7f1TLY4rhGrXZKDGS77+G/AQcAJ\nwO0YNtsvvEykVohNcU2yiuFKS1+sqp/2TqWRupjhtWPh4W0HMQPXqzbEjZLsDLyIq9YEV8/9/Ao6\n1/RzgcOq6q2TTZAvqKrvJzmakR//px2vqq5I8n6GvaUNsTbnDcCbkqxluPISwP0Yzrj10q6hFstz\nTfd6BcM/lNcxXF/1ecCbGF5sntU417TtC3xicvsyhh0vYNgZ4+COgTQ6Xwdu0z2ExqmqXsNw9aW7\nAK+f/LkL8NSqenXnbIvhGnGvxwPPrKqPJjmK4Vyppyf5NvBQ4C29403NTxnOjgPwY+DOwH8DNwGu\n2zWURuWlwOuSvAT4Mlc/wuBnHUNpPKrqvQxnK5w5hrjXnsCms2qtB244uf1RYPS/xS2hzwIPA77B\n8B/pb5I8FHgIM3DCdk3Fhyd/v5/5e09vuoLZSnkbR9ciyQ1ZsLV37L+oGeJePwT2mvx9GvBwht/2\n92dlnTP3MGDXye0jgSuA+zNE+ZVdQ2lUHtQ9gMYrya0Ydvg8iPlHXczEL2oevtQoyZHA+qo6Isnj\ngH8GfgTcEnhtVb2odUBJmgFJPsWwRfEo4CcsOOa8qj7dMddiGeIRSXJfhjXBU6vqQ93zTEuSjcCa\nqjpnwfKbAOesoL3HtQVJ7gI8A9iP4aplZyb5beAHVfXV3unUKcl64H5VdUr3LNvCvaYbJTlgcowk\nAFX1xap6PfDRJAc0jjZtuYbluzD/Ws1aoZI8jOGqS7cEHsxVO/HtB7ykay6Nxv8wvF7MJN8j7nUi\nsAY4Z8HyG0w+t6zXBCeXrYNhM9IzJ7/VbrITw7VmvzP1wTRGrwCeU1VvnhxrvslJwJ/1jKQRORw4\nMsmzFp5daxYY4l6bdiRY6CYsODxjmfrjyd8B/hDYOOdzG4AzgGdOeSaN052Bj2xm+c+AG095Fo3P\nCQxrxN9NchnDDp+/5CkudTVJPjC5WcA7Jv9wNtmJ4UXnP6c+2JRV1a0BkpwIPKaqzm8eSeP1M4bN\n0mcsWH5Phh0ctbId1j3A9jDEPTadpi/A+cw/VGkDcDLwD9MeqktVeWiKrs27gNcmeTzDL7CrkhzI\nsJfssa2Tqd2sX/TDvaYbTc4SdFRVrYTN0FuU5HbA49j81ZcOaRlKo5FkNXAc8ASGX2CvnPz9LuDg\nqtp4zV+tlSDJngynudwP+MvJ5TPvD/ykqv6nd7otM8SNklwHoKqunHx8C+DRwLeqatlvmt4kyaOA\nfwW+CtyLYe/Y/Rje8/lsVf1W43gakST7AfdgOOLjq1X1veaRNAJJ7gV8kmHv6TsBd5hcOOalwO2q\n6vc657s2Hr7U68NMdlhKsgewDngt8OkkT+kcbMpeDrysqvZnuOjDk4F9GC4EcVLfWL2S3CXJ0Un+\nPcmaybLfTnKP7tmmbfJzr66q06vqfVX1XiOsOY4C3lhV92B4DdnkPxjOzTBqhrjXWuBTk9uPAS4E\nbg48neHSgCvF7YH3TG5fDuxWVZcyBPpP26Zq5HGzV/Mu4Kwkfz/Z3CjNdS9gc+8Tn8lwTv9RM8S9\n9gB+Prn9MODfqupyhjjv1zbV9F3EVeeaPpOrLne3CrhRy0T9Nh03+zvMP6nJScB9WibqtSfDL6f7\nMWwx+n6SVya5Q/NcGodfsPnXijtw9fM0jI4h7vVD4P5Jdme44MOmKw3dGLikbarp+yLwgMntD3PV\n5e6OBT7fNlUvj5udo6ouqqpjq+qhDDv0HQ38BvDNJP/VO51G4ATgJUk2nV2rkuzDcBW7f+0aarEM\nca/XA//EcBzkj4HPTJYfwHBJwJXiOcAXJrdfCnwMeCzDFan+sGmmbpuOm11oxR83W1U/YQjxkQzX\nrb5n70Qagecy/IJ6LrAbwyGgpwEXAC9unGtR3Gu62WRvv72Bj1fV+smyRwE/r6rPtQ43BZNzbT8M\n+GJV/fTa7r9SJHk1wyk+H89wzeq1DKdDPQ44tqpe3jddnyQPAn6f4Rc1GK5P/I6qOrFvKo1Fkgcz\n/GJ2HeArVfWJ5pEWxRA3SXID4K5V9dnNfO7+DIcwrYgzTSW5lOFwgzO6ZxmLazhu9jrAO1mBx80m\neS3Dc3Fz4KPAO4APVNVlW/xCLXvL4bXUEDdJcj2GHZMePnfNN8ndgC8Bt6yq87rmm6YkXwReNCu/\nvU5Tkn256jf8FXvcbJLPMcT3PVX1s+55NB7L4bXUEDdK8k5gfVU9Y86yoxgOQF8xJ7FI8gjgVQyH\n5XyZBRe8WCkvvEnevtj7rsSzjU3exrgPmz/72vEtQ2kUZv211BA3SvJw4J+BW1TVhsmZtn4EHFZV\n7++dbnqSXDnnw7n/IANUVS3ry0FukuSDCxYdwLBJetOOe3dmWDP+zCy8uCylJLcHPgjsy/DvYiPD\n4W2XA5eN/eo62rFm/bXUiz70+jjD8W+PZtjp5CEMv+kvfEFe7p4G/C/zL4MIQ3T2nv44ParqNzfd\nTvIChn8bT9t0LvLJYW5vY2XtUb/JG4GvMJze8izg7gzX7f47ZmCvWO1wM/1a6hpxs8nesbevqt9O\ncjxwUVU9u3uuaUqyEVhTVecsWH4T4JyVskY8V5IzgYdU1bcWLL8T8MmqukXPZD2S/BQ4sKpOSXIB\ncJ+q+u7kCkx/W1V3bR5RzWb5tdQ14n7HA19OsjfwOwy/ya00Yf4m6U32AC6d8ixjsQewF8OhS3Ot\nYThOcqUJV53k5lyGY6y/y7D58TbX9EVaUWb2tdQQN6uqbyY5heGwlB9V1Ze6Z5qWJH8zuVnAkUnm\nnk1sJ4Ydc7429cHG4V+BY5M8j6tOdnI/hjMFjf49rx3gFOBuwPcZ9oR9/mRLytMZTtygFW6WX0sN\n8TgcD/w18KLuQabsLpO/A9yR+edU3sDwnuBR0x5qJP4IeB3DscSrJ8uuYHiPeCVdEGSTI4DdJ7df\nzHAq1BOB8xhOeiIgybeB21bVSn1tn8nXUt8jHoEkN2a4HOJbquqs7nmmLcmxwOFVdWH3LGMz2UFr\n0wVATt+045Z++f/m/PJF7JeSHAbcpKpe1j1Lh1l9LTXEkiQ18qIPkiQ1MsSSJDUyxCOR5NDuGcbE\n52M+n4/5fD7m8/mYb9aeD0M8HjP1D2cKfD7m8/mYz+djPp+P+Wbq+TDEkiQ1WvF7Te+cXWrXXx6e\n2OdyLmM1u3SPMRo+H/P5fMzn8zGfz8d8Y3k+LuL886rqZtd2v5V60Pcv7cru3DczcyY0SdKM+ES9\n7weLuZ+bpiVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSI\nJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJ\namSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJajTqECc5KcnR3XNIkrSjjDrEi5FkdfcMkiRt\nq9GGOMlxwIHAs5PU5M/Bk78fmeRLSTYAz0hyZZK1C77+6UnOS7Jzx/ySJC3Gqu4BtuBw4HbAd4AX\nTpbdafL3q4E/A04DLgJ+EzgEWDfn6w8B/qmqNkxlWkmStsFo14ir6gJgA3BJVZ1VVWcBGyeffmlV\nfayqvl9V5wL/ADwxya4ASe4I3A942+YeO8mhSdYlWXc5l+34H0aSpGsw2hBfi3ULPj6BIdqPmXx8\nCPClqjplc19cVcdU1dqqWruaXXbgmJIkbdmshvjiuR9U1eXA8cAhSVYBT+Ya1oYlSRqTMb9HDMNa\n7k6LvO9bgW8BzwKuB7x7Rw0lSdJSGXuIzwDuk2QfYD1bWIOvqu8mORl4LfDuqrpwGgNKkrQ9xr5p\n+iiGteJvAecCe1/L/d8G7IybpSVJM2LUa8RVdSqw/4LFx23hS9YA36uqz+ywoSRJWkKjDvFiJdkD\nuBXDscdHNI8jSdKijX3T9GIdDXwF+BzwluZZJElatGWxRlxVBwMHN48hSdJWWy5rxJIkzSRDLElS\nI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiND\nLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSo1XdA3TL6lWs\nutktuscYjR8+ad/uEUbl+mds7B5hVK7Y1d/d57rJCd/sHmFUNl54YfcIM8n/VZIkNTLEkiQ1MsSS\nJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1\nMsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLE\nkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDWauRAnOSnJ0d1zSJK0FGYuxJIkLSczFeIkxwEHAs9OUpM/\n+yQ5IMkXk1ya5Owkb0iyc/O4kiRdq5kKMXA48HngWGDN5M/lwL8DXwXuAfwB8ETgyKYZJUlatJkK\ncVVdAGwALqmqs6rqLOBZwE+AZ1XVt6vqQ8BfAIcl2W1zj5Pk0CTrkqzbcOUvpja/JEkLzVSIr8Ed\ngS9U1ZVzlp0M7AzcZnNfUFXHVNXaqlq783WuO40ZJUnarOUQ4i2p7gEkSdqSWQzxBmCnOR9/G7hf\nkrk/ywMm9zt9moNJkrS1ZjHEZwD3mewtfVPgzcBewJuT3DHJo4BXAUdX1SWNc0qSdK1mMcRHMazt\nfgs4F1gNPIJhj+mvAW8H/hl4YdeAkiQt1qruAbZWVZ0K7L9g8RnAfac/jSRJ22cW14glSVo2DLEk\nSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmN\nDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyx\nJEmNDLEkSY1WdQ/QrS6/givOOrt7jNH4lTdf1D3CqGy86226RxiVMx+we/cIo3LRH9+5e4RRudUb\nvtY9wrhcvLi7uUYsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0Ms\nSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElS\nI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUqMlCXGS6yR5S5KfJqkkZyT5\n0FI8tiRJy9mqJXqcRwJPAw4Cvg/8AsgSPbYkScvWUoX4NsCZVfWfS/R4i5Jk56raMM3vKUnSUtru\nTdNJjgPeAOw9Z7P0cXM3TSfZPcnxSdYnOTvJC5J8aPK1m+5zRpLnLnjsk5IcveA+L03y9iQ/B945\nWX7LJO9Ocv7kz4eT3HZ7fzZJkna0pXiP+HDg5cCPgDXAvTdzn9cBBwK/AzwYuBvwwG38fs8BvgOs\nBV6YZDfgRODSyffYHzgT+MTkc5IkjdZ2b5quqguSXARsrKqzAJKr3h5OsgdwCPCUqvr4ZNkfMIR7\nW3y6ql4z5/EPYXg/+mlVVZNlzwDOAR4NvHfhAyQ5FDgUYFdstSSpz1K9R7wl+wGrgS9tWlBVFyc5\nZRsfb92Cj+8F3Bq4aO4vAMBuk+99NVV1DHAMwPVz49rGOSRJ2m7TCPFiXcnV97RevZn7Xbzg4+sA\nXwOesJn7/mwJ5pIkaYeZxgk9TgcuZ857x5P3bu+84H7nMrzHvOk+uwJ3WMTjf4Vhr+3zquq0BX8M\nsSRp1HZ4iKtqPfB24NVJHpLk14C3Tr733M3CnwJ+P8lBSe40+ZrFrLG/EzgbOCHJgUluneSAJK9z\nz2lJ0thNa9P0c4HdgQ8A6xkOd9qTYU/nTY4E9gFOmNznCGCva3vgqrokyQHAq4B/AW4A/IRhT+rz\nl+wnkCRpB1iSEFfVUcBRcz4+eMHn1wNPnvwhyS7AnwIfmXOfC4EnLnjoNy94nH2u4fufzXBmL0mS\nZspU1oiT3AO4I8Oe09cDnj/5+z3T+P6SJI3VNPeafg5we+AKhr2cD6iqbT2WWJKkZWEqIa6qrzKc\nCUuSJM3h9YglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpk\niCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIgl\nSWpkiCVJarSqe4BRqOqeYDSuvPji7hFGJZ//evcIo3LLdTt3jzAqR5x6cvcIo/KX/+/J3SOMyymL\nu5trxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1\nMsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLE\nkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1WhYhTnJckg91zyFJ0tZa1T3AEjkcCECS\nk4BTquqw1okkSVqEZRHiqrqgewZJkrbFsghxkuOAmwLnAQcCByZ59uTTt66qM5pGkyRpi5ZFiOc4\nHLgd8B3ghZNl5/aNI0nSli2rEFfVBUk2AJdU1VnXdL8khwKHAuzKbtMaT5Kkq1kWe01vrao6pqrW\nVtXa1ezSPY4kaQVbkSGWJGkslmOINwA7dQ8hSdJiLMcQnwHcJ8k+SW6aZDn+jJKkZWI5RuoohrXi\nbzHsMb137ziSJF2zZbHXdFUdPOf2qcD+fdNIkrR4y3GNWJKkmWGIJUlqZIglSWpkiCVJamSIJUlq\nZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSI\nJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqtKp7AI1M0j2B\nRqyuuLx7hFF5/MnP6B5hVK7/oOt2jzAupyzubq4RS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS\n1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTI\nEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBL\nktRo2YU4yUFJKslNu2eRJOnaLLsQS5I0S0YX4iS7JPnrJGcnuTTJF5I8YPK5q63tJtlnsmxtkn2A\nEyefOney/Lip/xCSJC3S6EIMvAb4XeAQ4B7AN4CPJlmziK/9X+Cxk9t3AtYAh++IISVJWgqjCnGS\n3YE/Ap5fVR+uqm8DzwTOBp59bV9fVRuBn00+PKeqzqqqCzbzfQ5Nsi7Jusu5bAl/AkmSts6oQgzs\nB6wGPrdpwSSunwd+bam+SVUdU1Vrq2rtanZZqoeVJGmrjS3EW1LAlZPbmbN8dcMskiQtibGF+HRg\nA3D/TQuS7ATsD3wLOHeyeO77xXdf8BgbJn/vtINmlCRpyYwqxFV1MfB3wKuTPDLJHScf7wm8GTiN\nYYeslya5XZKHAS9e8DA/YFh7flSSmyXZY3o/gSRJW2dUIZ54PvAe4Fjga8Bdgd+oqjOr6nLgCcC+\nwNeBlwEvnPvFVfVj4CXAEQw7eR09vdElSdo6q7oHWKiqLgP+dPJnc5//T66+OToL7vMK4BU7ZEBJ\nkpbQGNeIJUlaMQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyx\nJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJ\njQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY1WdQ+gkanqnkCaGXd48XndI4zKsSe/u3uEUVnzt4u7\nn2vEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUy\nxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSS\nJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNVrVPUCH\nJIcChwLsym7N00iSVrIVuUZcVcdU1dqqWruaXbrHkSStYCsyxJIkjYUhliSp0bINcZLDknynew5J\nkrZk2YYYuClw++4hJEnakmUb4qp6aVWlew5JkrZk2YZYkqRZYIglSWpkiCVJamSIJUlqZIglSWpk\niCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIgl\nSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJarSqewBJ\nmlUbf3xm9wijcvOddu8eYSa5RixJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElS\nI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiND\nLElSI0MsSVIjQyxJUiNDLEmb1N3nAAAFt0lEQVRSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUqOZ\nCXGS5yY5o3sOSZKW0syEWJKk5WhJQpzk+kluuBSPtRXf82ZJdp3m95Qkaaltc4iT7JTk4UneBZwF\n3G2y/AZJjklyTpKLknw6ydo5X3dwkvVJHpLklCQXJzkxya0XPP6fJzlrct/jgT0WjPBI4KzJ97r/\ntv4ckiR12uoQJ7lTktcA/wu8B7gY+A3gM0kCfBi4JfBo4B7AZ4BPJVkz52F2AV4AHALsD9wQ+Ps5\n3+PxwCuBlwD3BL4LPGfBKO8Efg+4HvDxJKcl+b8Lgy5J0pgtKsRJbpLkT5J8GfgqcAfgcOAWVfX0\nqvpMVRXwIODuwOOq6ktVdVpV/SXwfeDJcx5yFfDsyX3+GzgKOGgScoA/Bf6xqt5SVadW1RHAl+bO\nVFVXVNVHquqJwC2Av5p8/+8lOSnJIUkWrkVv+nkOTbIuybrLuWwxT4EkSTvEYteI/xh4I3ApcLuq\n+q2q+pequnTB/e4F7AacO9mkvD7JeuDOwH5z7ndZVX13zsc/AXYGbjT5+I7A5xc89sKPf6mqLqyq\nt1fVg4B7A3sCbwMedw33P6aq1lbV2tXssoUfW5KkHWvVIu93DHA58BTglCT/BvwT8Mmq2jjnftcB\nzgYeuJnHuHDO7SsWfK7mfP1WS7ILw6bwJzG8d/xNhrXqE7bl8SRJmpZFha+qflJVR1TV7YFfB9YD\n7wZ+lOR1Se4+uetXGNZGr5xslp7755ytmOvbwP0WLJv3cQYPSPIWhp3F/hY4DbhXVd2zqt5YVedv\nxfeUJGnqtnoNtKq+UFV/BKxh2GR9O+C/kjwQ+ATwOeCEJI9Icusk+yd52eTzi/VG4KlJnp7ktkle\nANx3wX2eBHwMuD7wROBXq+p5VXXK1v5MkiR1Weym6aupqsuA9wHvS3JzYGNVVZJHMuzx/A/AzRk2\nVX8OOH4rHvs9SfYFjmB4z/kDwOuBg+fc7ZMMO4tdePVHkCRpNmTY2Xnlun5uXPfNQ7rHkDSDsmqb\n12WWpY/+cF33CKOy05rTvlxVa6/tfp7iUpKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKk\nRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaG\nWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEarugeQpFlVV1zRPcKo\nPHyvu3ePMDKnLeperhFLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LU\nyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQ\nS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS\n1MgQS5LUaFX3AB2SHAocCrAruzVPI0layVbkGnFVHVNVa6tq7Wp26R5HkrSCrcgQS5I0FoZYkqRG\nhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZY\nkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKk\nRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGqaruGVolORf4QfccwE2B87qHGBGf\nj/l8Pubz+ZjP52O+sTwft6qqm13bnVZ8iMciybqqWts9x1j4fMzn8zGfz8d8Ph/zzdrz4aZpSZIa\nGWJJkhoZ4vE4pnuAkfH5mM/nYz6fj/l8PuabqefD94glSWrkGrEkSY0MsSRJjQyxJEmNDLEkSY0M\nsSRJjf4/nX/okW6JJV8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# wrong translation\n",
    "translate(u'trata de averiguarlo.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RTe5P5ioMJwN"
   },
   "source": [
    "## 下一步\n",
    "- 下载不同的数据集以试验翻译，例如，英语到德语，或英语到法语。\n",
    "- 尝试对更大的数据集进行训练，或使用更多的时期"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "nmt_with_attention.ipynb",
   "provenance": [
    {
     "file_id": "https://github.com/tensorflow/docs/blob/master/site/en/r2/tutorials/text/nmt_with_attention.ipynb",
     "timestamp": 1564755135167
    }
   ],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
